-- Logs begin at Sat 2022-08-06 11:49:54 UTC, end at Sat 2022-08-06 13:22:08 UTC. -- Aug 06 12:16:39.622371 dv-cinder-851996-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 06 12:16:39.630958 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 06 12:16:39.632994 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Aug 6 12:16:39 2022] *** Aug 06 12:16:39.632994 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 06 12:16:39.632994 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 06 12:16:39.632994 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: nodename: dv-cinder-851996-3-iscsi Aug 06 12:16:39.632994 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: machine: x86_64 Aug 06 12:16:39.632994 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: clock source: unix Aug 06 12:16:39.632994 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: pcre jit disabled Aug 06 12:16:39.632994 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: detected number of CPU cores: 4 Aug 06 12:16:39.632994 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: current working directory: / Aug 06 12:16:39.632994 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: detected binary path: /usr/bin/uwsgi-core Aug 06 12:16:39.632994 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: your processes number limit is 47680 Aug 06 12:16:39.633697 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: your memory page size is 4096 bytes Aug 06 12:16:39.633697 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: detected max file descriptor number: 2048 Aug 06 12:16:39.633697 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: lock engine: pthread robust mutexes Aug 06 12:16:39.633697 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: thunder lock: enabled Aug 06 12:16:39.633697 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 06 12:16:39.633697 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 06 12:16:39.672284 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: Python main interpreter initialized at 0x5586b0ab79c0 Aug 06 12:16:39.672284 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: python threads support enabled Aug 06 12:16:39.672284 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: your server socket listen backlog is limited to 100 connections Aug 06 12:16:39.672284 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: your mercy for graceful operations on workers is 90 seconds Aug 06 12:16:39.672649 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: mapped 403077 bytes (393 KB) for 2 cores Aug 06 12:16:39.672719 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: *** Operational MODE: preforking *** Aug 06 12:16:39.672974 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: *** uWSGI is running in multiple interpreter mode *** Aug 06 12:16:39.672974 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: spawned uWSGI master process (pid: 80133) Aug 06 12:16:39.673171 dv-cinder-851996-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 06 12:16:39.676329 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: spawned uWSGI worker 1 (pid: 80144, cores: 1) Aug 06 12:16:39.676329 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: spawned uWSGI worker 2 (pid: 80145, cores: 1) Aug 06 12:16:39.676329 dv-cinder-851996-3-iscsi devstack@keystone.service[80133]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 06 12:16:45.102246 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 06 12:16:45.102246 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: from cryptography.utils import int_from_bytes Aug 06 12:16:45.103015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 06 12:16:45.103015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: from cryptography.utils import int_from_bytes Aug 06 12:16:45.105452 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 06 12:16:45.105452 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: from cryptography.utils import int_from_bytes Aug 06 12:16:45.105900 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 06 12:16:45.105900 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: from cryptography.utils import int_from_bytes Aug 06 12:16:45.152462 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 06 12:16:45.152650 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 06 12:16:45.152841 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] command line args: [] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 06 12:16:45.153111 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 06 12:16:45.153290 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ================================================================================ {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 06 12:16:45.153393 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 06 12:16:45.153492 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 06 12:16:45.153578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] admin_token = **** {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.153686 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] command line args: [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 06 12:16:45.153768 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] config_dir = [] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.153941 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.154016 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 06 12:16:45.154123 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] config_source = [] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.154210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ================================================================================ {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 06 12:16:45.154287 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] debug = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.154363 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] admin_token = **** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.154485 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.154744 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] config_dir = [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.154812 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.154869 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.154930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] config_source = [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.155074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.155174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] debug = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.155279 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.155387 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.155506 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.155606 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.155700 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.155807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] list_limit = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.155901 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] log_config_append = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.155992 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.156090 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.156189 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.156288 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] log_dir = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.156425 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] log_file = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.156525 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] list_limit = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.156619 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] log_config_append = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.156727 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.156821 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.156913 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.157034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] log_dir = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.157034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] log_file = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.157250 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.157330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.157444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.157444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.157640 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.157715 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.157838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.157912 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.158026 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.158108 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.158181 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.158305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.158388 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.158503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.158585 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.158753 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.158877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.158957 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.159074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.159181 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.159272 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.159414 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.159497 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.159648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.159730 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.159895 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.160141 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.160227 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.160374 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] publish_errors = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.160447 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.160572 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.160783 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.160926 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.161004 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.161110 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] publish_errors = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.161194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.161346 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.161470 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.161580 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.161676 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] standard_threads = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.161758 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.161828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.161994 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.162076 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.162215 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.162339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.162418 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] use_journal = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.162493 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] standard_threads = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.162562 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] use_json = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.162664 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.162741 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] use_stderr = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.162865 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] use_syslog = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.162946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.163068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.163142 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.163237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] use_journal = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.163333 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.163416 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] use_json = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.163570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.163646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] use_stderr = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.163742 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] use_syslog = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.163823 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.163972 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 06 12:16:45.164056 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.164229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.164324 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.164494 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.164600 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.164754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.164839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.164995 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.165077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.165243 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.165319 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.165484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.165567 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.165735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.165811 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.165973 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.166049 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.166215 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.166300 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.166451 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.167027 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.167227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.167313 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.167465 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.167553 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.167698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.167773 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.167948 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.168021 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.168196 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.168373 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.168468 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.168545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.168705 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.168779 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.168955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.169041 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.169207 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.169282 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.169455 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.169537 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.169693 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.169780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.169921 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.169996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.170152 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.170227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.170381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.170458 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.170614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.170696 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.170861 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.170942 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.171107 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.171182 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.171334 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.171419 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.171564 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.171639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.171810 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.171886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.172056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.172131 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.172288 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.172388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.172543 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.172619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.172831 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.172927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.173075 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.173310 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.173394 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.173540 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.173770 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.173850 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.174001 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.174227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.174378 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.174457 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.174574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.174756 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.174837 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.174984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.175068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.175218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.175302 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.175451 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.175523 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.175684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.175759 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.175917 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.175993 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.176162 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.176243 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.176416 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.176509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.176655 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.176732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.176890 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.176966 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.177125 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.177236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.177361 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.177463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.177597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.177697 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.177825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.177932 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.178057 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.178176 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.178292 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.178370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.178527 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.178608 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.178757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.178874 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.178998 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.179074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.179219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.179307 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.179453 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.179532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.179687 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.179764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.179922 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.179999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.180161 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.180240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.180419 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.180513 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.180676 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.180759 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.180913 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.180990 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.181145 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.181223 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.181380 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.181457 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.181615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.181695 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.181859 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.181937 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.182093 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.182172 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.182927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.183007 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.183171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.183247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.183408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.183483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.183642 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.183761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.183877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.183955 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.184111 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.184187 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.184371 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.184452 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.184615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.184692 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.184858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.184937 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.185096 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.185176 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.185328 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.185404 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.185563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.185638 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.185796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.185874 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.186030 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.186105 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.186261 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.186336 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.186481 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.186556 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.186709 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.186787 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.186939 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.187022 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.187176 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.187263 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.187390 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.187467 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.187636 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.187713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.187871 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.187956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.188102 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.188177 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.188356 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.188436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.188592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.188681 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.188830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.188908 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.189063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.189140 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.189303 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] auth.external = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.189388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.189535 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.189618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.189782 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.189860 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.190020 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.190110 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.190266 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] auth.password = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.190342 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.190497 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] auth.token = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.190573 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.190735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.190822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.190971 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.191063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] auth.external = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.191200 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.191277 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.191432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.191509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.191679 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.191766 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.191928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.192006 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] auth.password = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.192158 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] auth.token = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.192234 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.192422 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.192514 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] credential.caching = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.192663 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.192741 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.192908 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.192986 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.193137 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.193228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.193379 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.193456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.193624 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.193700 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.193852 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.193936 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.194090 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] credential.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.194168 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.194339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.194416 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.194573 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.194662 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.194824 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.194901 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.195062 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.195140 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.195307 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.195393 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.195544 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.195622 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.195811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.195888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.196064 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.196148 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.196352 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.196487 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] federation.caching = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.196565 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.196636 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.196790 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.196877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.197037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.197115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.197267 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.197353 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.197513 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.197602 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.197749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.197852 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.198021 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.198100 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] federation.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.198258 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.198345 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.198512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.198590 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.198748 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.198825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.198993 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.199078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.199228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.caching = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.199305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.199470 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.199546 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.199712 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.199790 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.199946 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.200023 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.200180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.200257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.200449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.200529 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.200681 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.200757 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.200917 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.200994 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.201163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.201241 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.201411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.201488 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.201641 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.201726 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.201872 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.201950 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.202100 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.202178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.202336 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.202411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.202576 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.202654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.202817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.202895 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.203065 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.203195 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.203284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.203357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.203517 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.203595 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.203755 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.203832 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.204003 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.204082 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.204229 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.204367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.204490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.204570 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.204720 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.204799 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.204956 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.205033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.205195 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.205273 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.205440 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.205519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.205688 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.205765 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.205919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.205996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.206156 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.206233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.206391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.206469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.206624 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.206700 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.206865 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.206943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.207107 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.207184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.207336 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.207419 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.207574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.207653 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.207806 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.207883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.208052 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.208129 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.208287 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.208381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.208551 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.208627 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.208787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.208866 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.209017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.209095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.209258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.209339 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.209490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.209574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.209719 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.209798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.209949 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.210034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.210188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.210265 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.210427 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.210506 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.210658 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.210735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.210894 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.210980 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.211131 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.211208 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.211360 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.211436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.211593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.211678 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.211842 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.211921 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.212071 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.212151 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.212332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.212428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.212578 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.212655 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.212817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.212894 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.213048 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.213131 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.213278 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.213355 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.213536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.213613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.213770 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.213855 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.214005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.214083 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.214232 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.214311 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.214477 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.214562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.214714 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.214791 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.214947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.215024 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.215184 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.215302 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.215423 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.215555 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.215662 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.215786 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.215893 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.216033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.216143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.216269 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.216528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.216605 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.216765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.216853 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.216999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.217077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.217240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.217317 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.217485 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.217570 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.217721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.217799 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.217959 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.218041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.218191 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.218276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.218427 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.218503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.218659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.218735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.218894 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.218979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.219130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.219208 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.219367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.219444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.219603 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.219687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.219835 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.219913 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.220068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.220145 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.220334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.220425 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] resource.caching = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.220565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.220643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.220804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.220881 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.221037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.221122 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.221277 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.221353 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.221511 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.221588 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.221744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.221830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.221985 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] resource.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.222061 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.222217 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.222293 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.222455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.222540 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.222690 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] role.caching = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.222767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.222935 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] role.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.223012 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.223165 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.223313 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.223409 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.223485 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.223650 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.223727 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.223890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.223974 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.224130 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.224208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.224395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.224475 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] role.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.224644 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] role.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.224722 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.224876 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.224954 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.225121 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.225197 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.225353 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.225432 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.225600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.225676 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.225842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.225918 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.226087 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.226166 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.226333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.226412 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.226583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.226743 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.226828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.226978 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.227055 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.227227 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.227305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.227474 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.227559 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.227720 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.227798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.227967 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.228044 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.228199 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.228408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.228490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.228650 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.228728 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.228904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.228982 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.229687 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.229766 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.229917 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.229995 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.230147 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.230233 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.230392 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.230469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.230642 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.230718 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.230888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.230974 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.231123 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.231201 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.231353 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.231428 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.231578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.231662 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.231810 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.231887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] token.caching = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.232042 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.232118 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.232286 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.232396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.232543 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.232620 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.232787 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.232863 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.233027 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.233112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.233259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.233336 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.233495 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] token.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.233572 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.233734 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.233817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.233979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.234056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.234214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.234291 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.234452 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.234536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.234685 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.234761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.234922 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.234998 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.235164 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.235249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.235392 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.235470 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.235634 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.235710 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.235870 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.235953 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.236110 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.236187 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.connection = **** {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.236370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.236464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.236620 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.236697 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.236868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.236947 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.237098 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.237175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.237334 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.237410 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.237570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.237648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.237804 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.237880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.connection = **** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.238038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.238124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.238282 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.238359 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.238517 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.238595 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.238758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.238835 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.238991 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.239068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.239497 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.239581 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.239728 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.239803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.239961 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.240036 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.240188 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.240274 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.240452 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.240543 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.240686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.240767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.240851 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 06 12:16:45.240926 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.241155 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.241392 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.241636 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.241869 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.242111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.242346 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 06 12:16:45.242516 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 06 12:16:45.269936 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.270108 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.270326 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.270628 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.270863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.271091 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.278579 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.279318 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.308842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.309068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.309586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.309821 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.360704 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.361600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.411025 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.411257 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.411869 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.412105 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.475201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.475422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.475643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.475859 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.475943 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.476071 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.476157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.476283 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.476389 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.476538 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.476761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.476980 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.499881 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.500115 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.500327 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.500418 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.500567 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.500650 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.500795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.500876 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.501015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.501106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.501227 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.501310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.501438 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.501657 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.512163 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.512556 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.524949 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.525134 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.525265 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.525399 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.525543 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.525689 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.525821 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.526448 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.526578 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.526714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.526846 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.526977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.527106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.527106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 06 12:16:45.543741 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.543949 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 06 12:16:45.544101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.544236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 06 12:16:45.544390 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.544518 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 06 12:16:45.544654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.544779 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 06 12:16:45.544901 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.545024 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.545147 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.545279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.545406 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.545528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.545660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.545783 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.555153 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.555427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.555555 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.555680 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.555803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.555950 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.556085 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.556214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.563422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.563699 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.568089 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.568389 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.568519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.568670 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.568795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.568948 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.569073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.569211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.569338 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.569478 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.569605 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.569731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.569866 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.569992 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.570118 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.570244 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.570370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.570507 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.570635 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.570762 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.570896 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.571023 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.594520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.594884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.596021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.596330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.596459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.596651 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.600955 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.601234 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.601368 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.601497 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.601624 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.601762 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.608133 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.608456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.608584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.608760 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.612586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.612863 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.612992 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.613126 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.613247 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.613417 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.613550 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.613674 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.613796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.613948 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.614074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.614467 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.614592 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.614715 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.614853 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.614977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.631239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.631386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.631512 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.631636 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.636031 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.636170 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.639200 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.639331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.639492 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.639619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.643910 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.644056 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.647400 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.647857 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.650576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.650871 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.651109 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.651237 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.651405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.651533 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.651686 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.651820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.651950 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.652078 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.652229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.652529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.667931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.668207 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.668493 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.668622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.668747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.668881 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.669057 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.669332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.675587 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.676222 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.677238 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.677510 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.677793 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.678068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.680351 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.680626 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.680911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.681047 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.681177 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.681303 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.681463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.681593 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.681729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.682004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.693439 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.693713 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.693991 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.694128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.694256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.694387 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.694512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.694641 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.694766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.694891 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.695024 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.695148 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.695319 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.695600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.710671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.711339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.714206 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.714882 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.717178 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.717854 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.719677 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.720286 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.721199 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.721484 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.721766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.721892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.722106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.722394 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.728184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.728894 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.729894 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.730549 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.732850 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.733137 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.733418 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.733501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.733691 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.733799 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.734086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.734373 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.741978 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.742259 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.742535 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.742725 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.743010 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.743290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.749911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.750190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.750465 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.750734 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.750826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 06 12:16:45.750998 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.751079 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.751259 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.751340 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.751524 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.751605 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.751787 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.751865 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.752052 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.752162 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.752339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.752466 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.752625 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.752737 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.752995 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.753076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.753253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.753331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.753521 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.753614 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.753777 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.754038 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.754306 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 06 12:16:45.786678 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5586b0ab79c0 pid: 80145 (default app) Aug 06 12:16:45.787619 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5586b0ab79c0 pid: 80144 (default app) Aug 06 12:16:45.789935 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0292bad-a33b-4a1f-ae2a-d8e8883270e0 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:16:45.790230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0292bad-a33b-4a1f-ae2a-d8e8883270e0 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:16:45.790503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0292bad-a33b-4a1f-ae2a-d8e8883270e0 None None] PATH_INFO: `/v3` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:16:45.795299 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Sat Aug 6 12:16:45 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:16:53.428080 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2ef1e-a577-481e-96cf-9f8fe7c01bc9 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:16:53.428653 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2ef1e-a577-481e-96cf-9f8fe7c01bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:16:53.428653 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2ef1e-a577-481e-96cf-9f8fe7c01bc9 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:16:53.433081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:16:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:16:53.438200 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c9e85-663b-4e1d-befa-308984c477c4 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:16:53.438578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c9e85-663b-4e1d-befa-308984c477c4 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:16:53.438743 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c9e85-663b-4e1d-befa-308984c477c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:16:53.507874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: INFO dbcounter [None req-f11c9e85-663b-4e1d-befa-308984c477c4 None None] Registered counter for database keystone Aug 06 12:16:53.527526 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG oslo_db.sqlalchemy.engines [None req-f11c9e85-663b-4e1d-befa-308984c477c4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80145) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 06 12:16:53.533614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writer thread running {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 06 12:16:53.709672 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG passlib.handlers.bcrypt [None req-f11c9e85-663b-4e1d-befa-308984c477c4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80145) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 06 12:16:53.709991 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG passlib.handlers.bcrypt [None req-f11c9e85-663b-4e1d-befa-308984c477c4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80145) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 06 12:16:53.764116 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-f11c9e85-663b-4e1d-befa-308984c477c4 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:16:53.765728 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f11c9e85-663b-4e1d-befa-308984c477c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:16:53.891307 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:16:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 06 12:16:53.924993 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2ef1e-a577-481e-96cf-9f8fe7c01bc9 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:16:53.994266 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: INFO dbcounter [None req-c2d2ef1e-a577-481e-96cf-9f8fe7c01bc9 None None] Registered counter for database keystone Aug 06 12:16:54.015772 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG oslo_db.sqlalchemy.engines [None req-c2d2ef1e-a577-481e-96cf-9f8fe7c01bc9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80144) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 06 12:16:54.022053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writer thread running {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 06 12:16:54.140552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2ef1e-a577-481e-96cf-9f8fe7c01bc9 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:16:54.144808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569cb5ee-7e52-43ee-a80b-59cfb9c7d094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:16:54.145441 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cb5ee-7e52-43ee-a80b-59cfb9c7d094 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:16:54.145724 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cb5ee-7e52-43ee-a80b-59cfb9c7d094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:16:54.145997 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cb5ee-7e52-43ee-a80b-59cfb9c7d094 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:16:54.154179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569cb5ee-7e52-43ee-a80b-59cfb9c7d094 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:16:54.209381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: WARNING keystone.common.rbac_enforcer.enforcer [None req-569cb5ee-7e52-43ee-a80b-59cfb9c7d094 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 06 12:16:54.209728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569cb5ee-7e52-43ee-a80b-59cfb9c7d094 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:16:54.219936 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Sat Aug 6 12:16:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 06 12:16:54.225876 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c9e85-663b-4e1d-befa-308984c477c4 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:16:54.240150 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c9e85-663b-4e1d-befa-308984c477c4 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:16:54.244559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0d917-7d04-4722-ad95-68e2bbdaf933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:16:54.245604 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0d917-7d04-4722-ad95-68e2bbdaf933 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:16:54.245886 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0d917-7d04-4722-ad95-68e2bbdaf933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:16:54.246158 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0d917-7d04-4722-ad95-68e2bbdaf933 admin admin] PATH_INFO: `/v3/projects/7ad154ab2bd64d7e8e712fcec9e1c94f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:16:54.247617 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf0d917-7d04-4722-ad95-68e2bbdaf933 admin admin] RBAC: Authorizing `identity:get_project(project_id=7ad154ab2bd64d7e8e712fcec9e1c94f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:16:54.297965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7cf0d917-7d04-4722-ad95-68e2bbdaf933 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 06 12:16:54.298303 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf0d917-7d04-4722-ad95-68e2bbdaf933 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:16:54.302857 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Sat Aug 6 12:16:54 2022] GET /identity/v3/projects/7ad154ab2bd64d7e8e712fcec9e1c94f => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:16:56.264774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdccbf7-941c-4491-a2b8-6758f6259fc4 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:16:56.265273 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdccbf7-941c-4491-a2b8-6758f6259fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:16:56.265273 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdccbf7-941c-4491-a2b8-6758f6259fc4 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:16:56.269409 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:16:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:16:56.274651 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8d065-b7d0-4c50-bf27-7d35c3e5d29c None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:16:56.275057 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8d065-b7d0-4c50-bf27-7d35c3e5d29c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:16:56.275205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8d065-b7d0-4c50-bf27-7d35c3e5d29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:16:56.295393 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-29e8d065-b7d0-4c50-bf27-7d35c3e5d29c None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:16:56.296913 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-29e8d065-b7d0-4c50-bf27-7d35c3e5d29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:16:56.304693 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:16:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 06 12:16:56.326674 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdccbf7-941c-4491-a2b8-6758f6259fc4 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:16:56.339066 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdccbf7-941c-4491-a2b8-6758f6259fc4 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:16:56.343186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07a69c-7d51-4670-96d1-32bab31e3e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:16:56.343777 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07a69c-7d51-4670-96d1-32bab31e3e76 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:16:56.344056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07a69c-7d51-4670-96d1-32bab31e3e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:16:56.344349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07a69c-7d51-4670-96d1-32bab31e3e76 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:16:56.351865 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07a69c-7d51-4670-96d1-32bab31e3e76 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:16:56.353654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07a69c-7d51-4670-96d1-32bab31e3e76 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:16:56.362912 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Sat Aug 6 12:16:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 12:16:56.368552 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e8d065-b7d0-4c50-bf27-7d35c3e5d29c None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:16:56.378131 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e8d065-b7d0-4c50-bf27-7d35c3e5d29c None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:16:56.382488 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e48e84f-87a6-4ece-9d98-8963abf9f0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:16:56.383111 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48e84f-87a6-4ece-9d98-8963abf9f0a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:16:56.383392 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48e84f-87a6-4ece-9d98-8963abf9f0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:16:56.383667 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48e84f-87a6-4ece-9d98-8963abf9f0a4 admin admin] PATH_INFO: `/v3/users/c9de148de73e459eb6456afdaf9fe213` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:16:56.385049 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e48e84f-87a6-4ece-9d98-8963abf9f0a4 admin admin] RBAC: Authorizing `identity:get_user(user_id=c9de148de73e459eb6456afdaf9fe213)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:16:56.386927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e48e84f-87a6-4ece-9d98-8963abf9f0a4 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:16:56.388488 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Sat Aug 6 12:16:56 2022] GET /identity/v3/users/c9de148de73e459eb6456afdaf9fe213 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:16:58.413155 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8335e7b-3112-4490-a0d9-44082425bdd5 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:16:58.414023 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8335e7b-3112-4490-a0d9-44082425bdd5 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:16:58.414023 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8335e7b-3112-4490-a0d9-44082425bdd5 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:16:58.415017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:16:58.421875 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175a829-6c00-41da-9a73-8654851df0a1 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:16:58.422299 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175a829-6c00-41da-9a73-8654851df0a1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:16:58.422564 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175a829-6c00-41da-9a73-8654851df0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:16:58.443628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-2175a829-6c00-41da-9a73-8654851df0a1 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:16:58.444875 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2175a829-6c00-41da-9a73-8654851df0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:16:58.471230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266f2b2-a9bb-4d8d-9669-b1a2ebe0ebb4 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:16:58.471510 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266f2b2-a9bb-4d8d-9669-b1a2ebe0ebb4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:16:58.471785 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266f2b2-a9bb-4d8d-9669-b1a2ebe0ebb4 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:16:58.473142 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:16:58.478034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f245c58-d72b-43bb-85db-33321d96be61 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:16:58.478308 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f245c58-d72b-43bb-85db-33321d96be61 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:16:58.478575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f245c58-d72b-43bb-85db-33321d96be61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:16:58.521015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG passlib.handlers.bcrypt [None req-4f245c58-d72b-43bb-85db-33321d96be61 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80144) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 06 12:16:58.521441 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG passlib.handlers.bcrypt [None req-4f245c58-d72b-43bb-85db-33321d96be61 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80144) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 06 12:16:58.526291 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:16:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:16:58.552495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175a829-6c00-41da-9a73-8654851df0a1 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:16:58.567938 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4f245c58-d72b-43bb-85db-33321d96be61 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:16:58.569547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4f245c58-d72b-43bb-85db-33321d96be61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:16:58.579548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:16:58.660537 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175a829-6c00-41da-9a73-8654851df0a1 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:16:58.680221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:16:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 12:16:58.697944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795852c6-18cd-43f3-86df-6d619b7865ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:16:58.698604 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795852c6-18cd-43f3-86df-6d619b7865ce None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:16:58.698888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795852c6-18cd-43f3-86df-6d619b7865ce None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:16:58.699173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795852c6-18cd-43f3-86df-6d619b7865ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:16:58.706230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f245c58-d72b-43bb-85db-33321d96be61 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:16:58.711091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795852c6-18cd-43f3-86df-6d619b7865ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:16:58.720085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-795852c6-18cd-43f3-86df-6d619b7865ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:16:58.720085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:16:58.720085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:16:58.720085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:16:58.720085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:16:58.720085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:16:58.720085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:16:58.720085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:16:58.720085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:16:58.720085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:16:58.720085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:16:58.720085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:16:58.720085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:16:58.720085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:16:58.720085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:16:58.720085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:16:58.721054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:16:58.721054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:16:58.721054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:16:58.721054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:16:58.721054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:16:58.721054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:16:58.721054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:16:58.721054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:16:58.721054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:16:58.721054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:16:58.721054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:16:58.721054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:16:58.721054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:16:58.721054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:16:58.721054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:16:58.721054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:16:58.722090 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:16:58.722090 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:16:58.722090 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:16:58.722090 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:16:58.722090 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:16:58.722090 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:16:58.722090 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:16:58.722090 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:16:58.722090 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:16:58.722090 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:16:58.722090 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:16:58.722090 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:16:58.722090 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:16:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:16:58.727327 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795852c6-18cd-43f3-86df-6d619b7865ce None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:16:58.810689 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:16:58.814334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f245c58-d72b-43bb-85db-33321d96be61 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:16:58.816709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:16:58.830211 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795852c6-18cd-43f3-86df-6d619b7865ce None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:16:58.857079 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661780b-41c1-4bb2-988f-a9ab9b81fa59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:16:58.858038 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661780b-41c1-4bb2-988f-a9ab9b81fa59 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:16:58.858383 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661780b-41c1-4bb2-988f-a9ab9b81fa59 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:16:58.858660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661780b-41c1-4bb2-988f-a9ab9b81fa59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:16:58.864918 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065cd4ed-86bf-4700-926d-386626c75ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:16:58.865553 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065cd4ed-86bf-4700-926d-386626c75ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:16:58.865836 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065cd4ed-86bf-4700-926d-386626c75ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:16:58.866111 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065cd4ed-86bf-4700-926d-386626c75ae9 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:16:58.866839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065cd4ed-86bf-4700-926d-386626c75ae9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:16:58.867142 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065cd4ed-86bf-4700-926d-386626c75ae9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:16:58.868604 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065cd4ed-86bf-4700-926d-386626c75ae9 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:16:58.880953 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 8/15] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:16:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:16:58.887295 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065cd4ed-86bf-4700-926d-386626c75ae9 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:16:58.928920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0661780b-41c1-4bb2-988f-a9ab9b81fa59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:16:58.930890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0661780b-41c1-4bb2-988f-a9ab9b81fa59 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:16:59.039537 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 8/16] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:16:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 12:16:59.045515 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661780b-41c1-4bb2-988f-a9ab9b81fa59 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:16:59.048772 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:16:59.097542 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065cd4ed-86bf-4700-926d-386626c75ae9 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:16:59.132200 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b38a9-7661-42e7-9f48-058089d6b8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:16:59.132976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b38a9-7661-42e7-9f48-058089d6b8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:16:59.133259 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b38a9-7661-42e7-9f48-058089d6b8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:16:59.133536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b38a9-7661-42e7-9f48-058089d6b8a0 None admin] PATH_INFO: `/v3/users/c9de148de73e459eb6456afdaf9fe213` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:16:59.134702 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b38a9-7661-42e7-9f48-058089d6b8a0 None admin] RBAC: Authorizing `identity:get_user(user_id=c9de148de73e459eb6456afdaf9fe213)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:16:59.136751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b38a9-7661-42e7-9f48-058089d6b8a0 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:16:59.138353 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 9/17] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:16:58 2022] GET /identity/v3/users/c9de148de73e459eb6456afdaf9fe213 => generated 247 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:16:59.143745 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b38a9-7661-42e7-9f48-058089d6b8a0 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:16:59.148820 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661780b-41c1-4bb2-988f-a9ab9b81fa59 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:16:59.158254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:16:59.184978 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5630f64-86ea-4430-902a-4287f5dc9170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:16:59.185756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5630f64-86ea-4430-902a-4287f5dc9170 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:16:59.186034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5630f64-86ea-4430-902a-4287f5dc9170 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:16:59.186308 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5630f64-86ea-4430-902a-4287f5dc9170 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:16:59.204332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5630f64-86ea-4430-902a-4287f5dc9170 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:16:59.205807 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5630f64-86ea-4430-902a-4287f5dc9170 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:16:59.224361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-b5630f64-86ea-4430-902a-4287f5dc9170 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:16:59.224361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:16:59.224361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:16:59.224361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:16:59.224361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:16:59.224361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:16:59.224361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:16:59.224361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:16:59.224361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:16:59.224361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:16:59.224361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:16:59.224361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:16:59.224361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:16:59.224361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:16:59.224361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:16:59.225205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:16:59.225205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:16:59.225205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:16:59.225205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:16:59.225205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:16:59.225205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:16:59.225205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:16:59.225205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:16:59.225205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:16:59.225205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:16:59.225205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:16:59.225205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:16:59.225205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:16:59.225205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:16:59.225205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:16:59.225205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:16:59.225205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:16:59.225998 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:16:59.225998 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:16:59.225998 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:16:59.225998 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:16:59.225998 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 9/18] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:16:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:16:59.231902 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5630f64-86ea-4430-902a-4287f5dc9170 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:16:59.246752 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b38a9-7661-42e7-9f48-058089d6b8a0 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:16:59.280692 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba209e0-721d-4073-b114-317aea419813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:16:59.281330 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba209e0-721d-4073-b114-317aea419813 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:16:59.281615 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba209e0-721d-4073-b114-317aea419813 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:16:59.281890 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba209e0-721d-4073-b114-317aea419813 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:16:59.283043 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba209e0-721d-4073-b114-317aea419813 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:16:59.284517 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba209e0-721d-4073-b114-317aea419813 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:16:59.286101 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 10/19] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:16:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:16:59.288857 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:16:59.291983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba209e0-721d-4073-b114-317aea419813 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:16:59.332948 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5630f64-86ea-4430-902a-4287f5dc9170 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:16:59.366273 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffdf2e-e14e-4bc2-ab13-8471a8b6572a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:16:59.366903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffdf2e-e14e-4bc2-ab13-8471a8b6572a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:16:59.367183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffdf2e-e14e-4bc2-ab13-8471a8b6572a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:16:59.367454 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffdf2e-e14e-4bc2-ab13-8471a8b6572a None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:16:59.368146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edffdf2e-e14e-4bc2-ab13-8471a8b6572a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:16:59.368473 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edffdf2e-e14e-4bc2-ab13-8471a8b6572a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:16:59.370021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edffdf2e-e14e-4bc2-ab13-8471a8b6572a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:16:59.371375 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:16:59.388246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 10/20] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:16:59.391295 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba209e0-721d-4073-b114-317aea419813 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:16:59.427654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60fc3d1-058d-4eea-907e-96bb6f0dcdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:16:59.428294 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fc3d1-058d-4eea-907e-96bb6f0dcdba None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:16:59.428575 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fc3d1-058d-4eea-907e-96bb6f0dcdba None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:16:59.428847 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fc3d1-058d-4eea-907e-96bb6f0dcdba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:16:59.429650 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60fc3d1-058d-4eea-907e-96bb6f0dcdba None admin] RBAC: Adding query filter params (role.id=144c8425688b409384e1e8ed8279d88e, scope.domain.id=default, user.id=c9de148de73e459eb6456afdaf9fe213) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:16:59.429961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60fc3d1-058d-4eea-907e-96bb6f0dcdba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:16:59.431452 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60fc3d1-058d-4eea-907e-96bb6f0dcdba None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:16:59.439402 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 11/21] 10.222.0.20 () {56 vars in 1322 bytes} [Sat Aug 6 12:16:59 2022] GET /identity/v3/role_assignments?user.id=c9de148de73e459eb6456afdaf9fe213&scope.domain.id=default&role.id=144c8425688b409384e1e8ed8279d88e => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:01.686246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f445d4f-a54d-4f0a-80ed-619a6cf7aef6 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:01.686764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f445d4f-a54d-4f0a-80ed-619a6cf7aef6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:01.686764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f445d4f-a54d-4f0a-80ed-619a6cf7aef6 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:01.688400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 11/22] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:01.693759 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247efef-5e8f-4358-8844-47bb2f0273f6 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:01.694152 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247efef-5e8f-4358-8844-47bb2f0273f6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:01.694316 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247efef-5e8f-4358-8844-47bb2f0273f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:01.725276 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-f247efef-5e8f-4358-8844-47bb2f0273f6 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:01.726471 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f247efef-5e8f-4358-8844-47bb2f0273f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:01.825710 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd663c6-f668-4364-8ad3-3fdf8b91c8df None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:01.826004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd663c6-f668-4364-8ad3-3fdf8b91c8df None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:01.826251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd663c6-f668-4364-8ad3-3fdf8b91c8df None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:01.827583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 12/23] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:01.837024 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255136b4-3bed-4bd0-a9e5-3de146912db7 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:01.837303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255136b4-3bed-4bd0-a9e5-3de146912db7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:01.837579 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255136b4-3bed-4bd0-a9e5-3de146912db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:01.847203 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 12/24] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 12:17:01.864487 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-255136b4-3bed-4bd0-a9e5-3de146912db7 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:01.865935 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-255136b4-3bed-4bd0-a9e5-3de146912db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:01.897171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f247efef-5e8f-4358-8844-47bb2f0273f6 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:01.978097 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 13/25] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:02.008723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255136b4-3bed-4bd0-a9e5-3de146912db7 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:02.023007 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:02.068663 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f247efef-5e8f-4358-8844-47bb2f0273f6 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:02.089309 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:02.138774 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b992a5-93ef-4d50-8493-7b2234d4a879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:02.139434 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b992a5-93ef-4d50-8493-7b2234d4a879 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:02.139716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b992a5-93ef-4d50-8493-7b2234d4a879 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:02.140034 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b992a5-93ef-4d50-8493-7b2234d4a879 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:02.163070 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b992a5-93ef-4d50-8493-7b2234d4a879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:02.164604 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b992a5-93ef-4d50-8493-7b2234d4a879 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:02.184606 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255136b4-3bed-4bd0-a9e5-3de146912db7 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:02.194931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-76b992a5-93ef-4d50-8493-7b2234d4a879 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:17:02.194931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:02.194931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:02.194931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:02.194931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:02.194931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:02.194931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:02.194931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:02.194931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:02.194931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:02.194931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:02.194931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:02.194931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:17:02.194931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:17:02.194931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:17:02.195757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:17:02.195757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:02.195757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:02.195757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:02.195757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:02.195757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:02.195757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:02.195757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:02.195757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:02.195757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:02.195757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:02.195757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:02.195757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:02.195757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:02.195757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:02.195757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:02.195757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:02.196582 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:17:02.196582 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:17:02.196582 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:17:02.196582 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:17:02.201966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 13/26] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:17:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:17:02.203283 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a0517-22e9-4476-9300-ce3ad23509ad None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:02.203573 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a0517-22e9-4476-9300-ce3ad23509ad None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:02.203851 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a0517-22e9-4476-9300-ce3ad23509ad None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:02.205188 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 14/27] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:02.209707 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a0517-22e9-4476-9300-ce3ad23509ad None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:02.263277 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af47db5-1b8f-4f48-b9de-7b68e7b48731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:02.263865 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af47db5-1b8f-4f48-b9de-7b68e7b48731 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:02.264145 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af47db5-1b8f-4f48-b9de-7b68e7b48731 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:02.264447 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af47db5-1b8f-4f48-b9de-7b68e7b48731 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:02.287696 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af47db5-1b8f-4f48-b9de-7b68e7b48731 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:02.302825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-1af47db5-1b8f-4f48-b9de-7b68e7b48731 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:17:02.302825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:02.302825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:02.302825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:02.302825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:02.302825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:02.302825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:02.302825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:02.302825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:02.302825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:02.302825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:02.302825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:02.302825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:02.302825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:02.302825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:02.302825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:02.303713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:02.303713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:02.303713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:02.303713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:02.303713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:02.303713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:02.303713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:02.303713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:02.303713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:02.303713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:02.303713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:02.303713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:02.303713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:02.303713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:02.303713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:02.303713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:02.304477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:02.304477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:02.304477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:02.304477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:02.304477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:02.304477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:02.304477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:02.304477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:02.304477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:02.304477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:02.304477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:17:02.304477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:02.306880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 14/28] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:17:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:02.308475 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2735e-4759-4420-884f-c150f3fe52b9 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:02.308757 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2735e-4759-4420-884f-c150f3fe52b9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:02.309031 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2735e-4759-4420-884f-c150f3fe52b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:02.339490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4aa2735e-4759-4420-884f-c150f3fe52b9 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:02.342849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4aa2735e-4759-4420-884f-c150f3fe52b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:02.395158 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a0517-22e9-4476-9300-ce3ad23509ad None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:02.428881 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:02.452446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ab9b6-10be-4458-a5dd-e69eb2a57d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:02.453070 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ab9b6-10be-4458-a5dd-e69eb2a57d7d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:02.453390 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ab9b6-10be-4458-a5dd-e69eb2a57d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:02.453665 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ab9b6-10be-4458-a5dd-e69eb2a57d7d None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:02.454411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ab9b6-10be-4458-a5dd-e69eb2a57d7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:02.455113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ab9b6-10be-4458-a5dd-e69eb2a57d7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:02.458343 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ab9b6-10be-4458-a5dd-e69eb2a57d7d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:02.496166 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 15/29] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:02.497087 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ab9b6-10be-4458-a5dd-e69eb2a57d7d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:02.503738 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 15/30] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:02.504723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2735e-4759-4420-884f-c150f3fe52b9 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:02.554518 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:02.671647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ab9b6-10be-4458-a5dd-e69eb2a57d7d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:02.712380 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c4fbc7-23d4-415e-8d26-331839ba0129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:02.714933 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c4fbc7-23d4-415e-8d26-331839ba0129 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:02.715367 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c4fbc7-23d4-415e-8d26-331839ba0129 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:02.715778 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c4fbc7-23d4-415e-8d26-331839ba0129 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:02.716957 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c4fbc7-23d4-415e-8d26-331839ba0129 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:02.717417 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c4fbc7-23d4-415e-8d26-331839ba0129 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:02.719664 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c4fbc7-23d4-415e-8d26-331839ba0129 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:02.731006 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 16/31] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:02.736922 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c4fbc7-23d4-415e-8d26-331839ba0129 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:02.742726 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2735e-4759-4420-884f-c150f3fe52b9 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:02.779084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd2f923-7ffe-4012-9620-6ee12c788b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:02.779973 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd2f923-7ffe-4012-9620-6ee12c788b5d None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:02.780434 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd2f923-7ffe-4012-9620-6ee12c788b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:02.780857 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd2f923-7ffe-4012-9620-6ee12c788b5d None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:02.781912 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd2f923-7ffe-4012-9620-6ee12c788b5d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:02.783717 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd2f923-7ffe-4012-9620-6ee12c788b5d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:02.792701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:02.852011 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c4fbc7-23d4-415e-8d26-331839ba0129 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:02.892123 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c2cd76-c3ee-4abc-b862-a036e5d28db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:02.892879 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2cd76-c3ee-4abc-b862-a036e5d28db2 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:02.893167 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2cd76-c3ee-4abc-b862-a036e5d28db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:02.893443 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2cd76-c3ee-4abc-b862-a036e5d28db2 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:02.894245 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c2cd76-c3ee-4abc-b862-a036e5d28db2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:02.895640 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c2cd76-c3ee-4abc-b862-a036e5d28db2 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:02.956670 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 16/32] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Aug 6 12:17:02 2022] POST /identity/v3/projects => generated 301 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:17:02.958771 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb0eaa-48dc-431b-8012-ce8f779333ea None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:02.959257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb0eaa-48dc-431b-8012-ce8f779333ea None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:02.959705 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb0eaa-48dc-431b-8012-ce8f779333ea None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:02.961228 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 17/33] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:02.962085 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb0eaa-48dc-431b-8012-ce8f779333ea None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:03.036628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 17/34] 10.222.0.20 () {60 vars in 1131 bytes} [Sat Aug 6 12:17:02 2022] POST /identity/v3/roles => generated 219 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:17:03.039020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff3861-6098-46f4-acfe-154ae8050468 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:03.039323 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff3861-6098-46f4-acfe-154ae8050468 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:03.039656 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff3861-6098-46f4-acfe-154ae8050468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:03.065471 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-cdff3861-6098-46f4-acfe-154ae8050468 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:03.067161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-cdff3861-6098-46f4-acfe-154ae8050468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:03.151597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:03.167530 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 18/35] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:03.168861 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875754d7-0bc0-49df-9902-73d9b207c223 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:03.169269 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875754d7-0bc0-49df-9902-73d9b207c223 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:03.169646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875754d7-0bc0-49df-9902-73d9b207c223 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:03.171222 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 19/36] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:03.171445 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb0eaa-48dc-431b-8012-ce8f779333ea None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:03.176666 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebe00e-d2f1-4bba-82bc-f320e0d02d19 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:03.177057 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebe00e-d2f1-4bba-82bc-f320e0d02d19 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:03.177431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebe00e-d2f1-4bba-82bc-f320e0d02d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:03.196647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-17ebe00e-d2f1-4bba-82bc-f320e0d02d19 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:03.198561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-17ebe00e-d2f1-4bba-82bc-f320e0d02d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:03.220264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8733732-7e46-43d0-bcd2-7b897940435a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:03.221516 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8733732-7e46-43d0-bcd2-7b897940435a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:03.221976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8733732-7e46-43d0-bcd2-7b897940435a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:03.222349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8733732-7e46-43d0-bcd2-7b897940435a None admin] PATH_INFO: `/v3/users/c9de148de73e459eb6456afdaf9fe213` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:03.224559 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8733732-7e46-43d0-bcd2-7b897940435a None admin] RBAC: Authorizing `identity:get_user(user_id=c9de148de73e459eb6456afdaf9fe213)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:03.226189 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8733732-7e46-43d0-bcd2-7b897940435a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:03.227876 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 18/37] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:02 2022] GET /identity/v3/users/c9de148de73e459eb6456afdaf9fe213 => generated 247 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:03.228699 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8733732-7e46-43d0-bcd2-7b897940435a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:03.247999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:03.306606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 20/38] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:03.309275 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebe00e-d2f1-4bba-82bc-f320e0d02d19 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:03.378072 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8733732-7e46-43d0-bcd2-7b897940435a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:03.415213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba08c69-1791-42e8-a943-48be038f9166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:03.415758 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba08c69-1791-42e8-a943-48be038f9166 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:03.416037 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba08c69-1791-42e8-a943-48be038f9166 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:03.416321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba08c69-1791-42e8-a943-48be038f9166 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:03.417668 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba08c69-1791-42e8-a943-48be038f9166 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:03.419040 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba08c69-1791-42e8-a943-48be038f9166 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:03.428218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebe00e-d2f1-4bba-82bc-f320e0d02d19 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:03.434599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 19/39] 10.222.0.20 () {60 vars in 1131 bytes} [Sat Aug 6 12:17:03 2022] POST /identity/v3/roles => generated 225 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:17:03.435804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ddd7ef-f743-4025-843e-12beabf618a6 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:03.436092 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ddd7ef-f743-4025-843e-12beabf618a6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:03.436407 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ddd7ef-f743-4025-843e-12beabf618a6 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:03.437667 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 20/40] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:03.438389 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ddd7ef-f743-4025-843e-12beabf618a6 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:03.463066 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:03.476991 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c53c89-f624-4915-8c64-92ac8ff631d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:03.477710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c53c89-f624-4915-8c64-92ac8ff631d3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:03.477992 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c53c89-f624-4915-8c64-92ac8ff631d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:03.478265 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c53c89-f624-4915-8c64-92ac8ff631d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:03.480083 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c53c89-f624-4915-8c64-92ac8ff631d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:03.481889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c53c89-f624-4915-8c64-92ac8ff631d3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:03.483569 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 21/41] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:17:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:03.484778 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c445b2-22fb-4a4b-a2c9-0cdb20418fc1 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:03.485060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c445b2-22fb-4a4b-a2c9-0cdb20418fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:03.485332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c445b2-22fb-4a4b-a2c9-0cdb20418fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:03.505634 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-21c445b2-22fb-4a4b-a2c9-0cdb20418fc1 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:03.507172 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-21c445b2-22fb-4a4b-a2c9-0cdb20418fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:03.540020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:03.577937 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ddd7ef-f743-4025-843e-12beabf618a6 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:03.584864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 22/42] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:03.585714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c445b2-22fb-4a4b-a2c9-0cdb20418fc1 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:03.628581 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e106b-2a67-4a3e-bb24-6a541dffd56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:03.629240 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e106b-2a67-4a3e-bb24-6a541dffd56f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:03.629544 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e106b-2a67-4a3e-bb24-6a541dffd56f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:03.629823 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e106b-2a67-4a3e-bb24-6a541dffd56f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:03.633055 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e106b-2a67-4a3e-bb24-6a541dffd56f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:03.634506 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e106b-2a67-4a3e-bb24-6a541dffd56f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:03.636324 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 21/43] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:17:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:03.636999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e106b-2a67-4a3e-bb24-6a541dffd56f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:03.700940 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c445b2-22fb-4a4b-a2c9-0cdb20418fc1 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:03.723850 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:03.734078 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56763b-136b-4c83-86be-3dc2c51383b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:03.734753 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56763b-136b-4c83-86be-3dc2c51383b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:03.735035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56763b-136b-4c83-86be-3dc2c51383b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:03.735317 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56763b-136b-4c83-86be-3dc2c51383b0 None admin] PATH_INFO: `/v3/domains/default/users/c9de148de73e459eb6456afdaf9fe213/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:03.737309 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56763b-136b-4c83-86be-3dc2c51383b0 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c9de148de73e459eb6456afdaf9fe213, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:03.738759 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56763b-136b-4c83-86be-3dc2c51383b0 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:03.740790 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e106b-2a67-4a3e-bb24-6a541dffd56f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:03.753735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 23/44] 10.222.0.20 () {58 vars in 1361 bytes} [Sat Aug 6 12:17:03 2022] PUT /identity/v3/domains/default/users/c9de148de73e459eb6456afdaf9fe213/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:17:03.754597 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56763b-136b-4c83-86be-3dc2c51383b0 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:03.784311 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb87350-964a-4618-87e8-38e427dc4df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:03.784963 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb87350-964a-4618-87e8-38e427dc4df3 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:03.785249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb87350-964a-4618-87e8-38e427dc4df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:03.785520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb87350-964a-4618-87e8-38e427dc4df3 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:03.785732 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:03.786562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb87350-964a-4618-87e8-38e427dc4df3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:03.787918 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb87350-964a-4618-87e8-38e427dc4df3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:03.804712 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 22/45] 10.222.0.20 () {60 vars in 1131 bytes} [Sat Aug 6 12:17:03 2022] POST /identity/v3/roles => generated 223 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:17:03.882316 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56763b-136b-4c83-86be-3dc2c51383b0 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:03.924013 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d65a40-1b10-4883-8d78-902d034748d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:03.924700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d65a40-1b10-4883-8d78-902d034748d8 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:03.924987 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d65a40-1b10-4883-8d78-902d034748d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:03.925268 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d65a40-1b10-4883-8d78-902d034748d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:03.925972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d65a40-1b10-4883-8d78-902d034748d8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:03.927464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d65a40-1b10-4883-8d78-902d034748d8 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:03.960292 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 24/46] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Aug 6 12:17:03 2022] POST /identity/v3/projects => generated 312 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:17:05.768921 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa803b-c7e9-4f73-945b-c8084c31f1f7 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:05.769468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa803b-c7e9-4f73-945b-c8084c31f1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:05.769468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa803b-c7e9-4f73-945b-c8084c31f1f7 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:05.770823 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 23/47] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:05.776438 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbac6eb-9fde-4756-b6b1-f3fb747bf99b None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:05.776841 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbac6eb-9fde-4756-b6b1-f3fb747bf99b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:05.776995 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbac6eb-9fde-4756-b6b1-f3fb747bf99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:05.794906 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-4fbac6eb-9fde-4756-b6b1-f3fb747bf99b None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:05.796078 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4fbac6eb-9fde-4756-b6b1-f3fb747bf99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:05.860094 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 25/48] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:05.882574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa803b-c7e9-4f73-945b-c8084c31f1f7 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:05.972886 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf46b7-27c5-40e2-b1ae-aa3d32965fba None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:05.973200 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf46b7-27c5-40e2-b1ae-aa3d32965fba None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:05.973457 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf46b7-27c5-40e2-b1ae-aa3d32965fba None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:05.974972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 26/49] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:17:05.980015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bdfc2-90a4-405c-af9d-9613804860ff None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:05.980321 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bdfc2-90a4-405c-af9d-9613804860ff None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:05.980609 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bdfc2-90a4-405c-af9d-9613804860ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:05.983360 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa803b-c7e9-4f73-945b-c8084c31f1f7 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:06.000125 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-b14bdfc2-90a4-405c-af9d-9613804860ff None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:06.001342 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b14bdfc2-90a4-405c-af9d-9613804860ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:06.003354 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:06.021713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d1e3f-b09f-4dc5-91e0-84d4e4c45331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:06.022343 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d1e3f-b09f-4dc5-91e0-84d4e4c45331 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:06.022649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d1e3f-b09f-4dc5-91e0-84d4e4c45331 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:06.022931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d1e3f-b09f-4dc5-91e0-84d4e4c45331 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:06.034075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d1e3f-b09f-4dc5-91e0-84d4e4c45331 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:06.039058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-6b6d1e3f-b09f-4dc5-91e0-84d4e4c45331 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:17:06.039058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:06.039058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:06.039058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:06.039058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:06.039058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:06.039058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:06.039058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:06.039058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:06.039058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:06.039058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:06.039058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:06.039058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:06.039058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:06.039058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:06.039058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:06.040111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:06.040111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:06.040111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:06.040111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:06.040111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:06.040111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:06.040111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:06.040111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:06.040111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:06.040111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:06.040111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:06.040111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:06.040111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:06.040111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:06.040111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:06.040111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:06.040907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:06.040907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:06.040907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:06.040907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:06.040907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:06.040907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:06.040907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:06.040907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:06.040907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:06.040907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:06.040907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:17:06.040907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:06.040907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 24/50] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:17:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:06.046102 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d1e3f-b09f-4dc5-91e0-84d4e4c45331 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:06.049469 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:06.069442 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 27/51] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 12:17:06.091945 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bdfc2-90a4-405c-af9d-9613804860ff None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:06.144787 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d1e3f-b09f-4dc5-91e0-84d4e4c45331 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:06.178733 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38443345-98ac-41da-b6ac-3067f2f23a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:06.179347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38443345-98ac-41da-b6ac-3067f2f23a89 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:06.179628 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38443345-98ac-41da-b6ac-3067f2f23a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:06.179903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38443345-98ac-41da-b6ac-3067f2f23a89 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:06.180661 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38443345-98ac-41da-b6ac-3067f2f23a89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:06.180969 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38443345-98ac-41da-b6ac-3067f2f23a89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:06.182426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38443345-98ac-41da-b6ac-3067f2f23a89 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:06.189244 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bdfc2-90a4-405c-af9d-9613804860ff None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:06.194004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 25/52] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:06.200028 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38443345-98ac-41da-b6ac-3067f2f23a89 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:06.225447 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3caa627-a2d9-4753-9d2a-93725260c07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:06.226123 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3caa627-a2d9-4753-9d2a-93725260c07b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:06.226408 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3caa627-a2d9-4753-9d2a-93725260c07b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:06.226695 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3caa627-a2d9-4753-9d2a-93725260c07b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:06.227929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3caa627-a2d9-4753-9d2a-93725260c07b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:06.229463 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3caa627-a2d9-4753-9d2a-93725260c07b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:06.231050 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 28/53] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:17:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:06.236672 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3caa627-a2d9-4753-9d2a-93725260c07b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:06.245130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:06.269467 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:06.304014 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38443345-98ac-41da-b6ac-3067f2f23a89 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:06.334888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3caa627-a2d9-4753-9d2a-93725260c07b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:06.338259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c255cc02-5655-4210-b6ca-19cea6a02414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:06.338955 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255cc02-5655-4210-b6ca-19cea6a02414 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:06.339236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255cc02-5655-4210-b6ca-19cea6a02414 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:06.339510 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255cc02-5655-4210-b6ca-19cea6a02414 None admin] PATH_INFO: `/v3/users/c9de148de73e459eb6456afdaf9fe213` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:06.340716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c255cc02-5655-4210-b6ca-19cea6a02414 None admin] RBAC: Authorizing `identity:get_user(user_id=c9de148de73e459eb6456afdaf9fe213)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:06.342170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c255cc02-5655-4210-b6ca-19cea6a02414 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:06.343762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 26/54] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:06 2022] GET /identity/v3/users/c9de148de73e459eb6456afdaf9fe213 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:06.349076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c255cc02-5655-4210-b6ca-19cea6a02414 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:06.368941 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f6ee1-ccbe-45dd-a57a-b288d82863ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:06.369552 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f6ee1-ccbe-45dd-a57a-b288d82863ec None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:06.369906 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f6ee1-ccbe-45dd-a57a-b288d82863ec None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:06.370243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f6ee1-ccbe-45dd-a57a-b288d82863ec None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:06.370919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f6ee1-ccbe-45dd-a57a-b288d82863ec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:06.372411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f6ee1-ccbe-45dd-a57a-b288d82863ec None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:06.399933 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 29/55] 10.222.0.20 () {60 vars in 1140 bytes} [Sat Aug 6 12:17:06 2022] POST /identity/v3/projects => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 12:17:06.454638 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c255cc02-5655-4210-b6ca-19cea6a02414 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:06.491472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899d5d9-0a9b-4923-bf0c-a57a091b1af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:06.492126 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899d5d9-0a9b-4923-bf0c-a57a091b1af1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:06.492438 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899d5d9-0a9b-4923-bf0c-a57a091b1af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:06.492719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899d5d9-0a9b-4923-bf0c-a57a091b1af1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:06.493918 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899d5d9-0a9b-4923-bf0c-a57a091b1af1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:06.495417 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899d5d9-0a9b-4923-bf0c-a57a091b1af1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:06.497095 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 27/56] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:17:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:06.503091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f6ee1-ccbe-45dd-a57a-b288d82863ec None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:06.563881 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:06.606852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f6ee1-ccbe-45dd-a57a-b288d82863ec None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:06.642800 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a808aa-a97a-4832-90c9-2aa45fb3bae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:06.643414 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a808aa-a97a-4832-90c9-2aa45fb3bae2 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:06.643691 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a808aa-a97a-4832-90c9-2aa45fb3bae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:06.643966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a808aa-a97a-4832-90c9-2aa45fb3bae2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:06.644787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a808aa-a97a-4832-90c9-2aa45fb3bae2 None admin] RBAC: Adding query filter params (role.id=144c8425688b409384e1e8ed8279d88e, scope.domain.id=default, user.id=c9de148de73e459eb6456afdaf9fe213) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:06.645099 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a808aa-a97a-4832-90c9-2aa45fb3bae2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:06.646544 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a808aa-a97a-4832-90c9-2aa45fb3bae2 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:06.652710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 30/57] 10.222.0.20 () {56 vars in 1322 bytes} [Sat Aug 6 12:17:06 2022] GET /identity/v3/role_assignments?user.id=c9de148de73e459eb6456afdaf9fe213&scope.domain.id=default&role.id=144c8425688b409384e1e8ed8279d88e => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:08.390478 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04663512-2cae-4979-a96e-2d5db4d7dadb None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:08.390982 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04663512-2cae-4979-a96e-2d5db4d7dadb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:08.391063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04663512-2cae-4979-a96e-2d5db4d7dadb None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:08.392399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 28/58] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:08.397520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e176ac9-5fe2-47be-839b-1efff16560c4 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:08.397944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e176ac9-5fe2-47be-839b-1efff16560c4 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:08.398067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e176ac9-5fe2-47be-839b-1efff16560c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:08.416138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-9e176ac9-5fe2-47be-839b-1efff16560c4 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:08.417360 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9e176ac9-5fe2-47be-839b-1efff16560c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:08.480317 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 31/59] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:08.504177 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04663512-2cae-4979-a96e-2d5db4d7dadb None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:08.507051 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:08.602611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04663512-2cae-4979-a96e-2d5db4d7dadb None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:08.634894 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b28df4-7b26-4fe5-af79-c0db1c5d9d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:08.635586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b28df4-7b26-4fe5-af79-c0db1c5d9d45 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:08.635862 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b28df4-7b26-4fe5-af79-c0db1c5d9d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:08.636131 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b28df4-7b26-4fe5-af79-c0db1c5d9d45 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:08.637285 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b28df4-7b26-4fe5-af79-c0db1c5d9d45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:08.638707 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b28df4-7b26-4fe5-af79-c0db1c5d9d45 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:08.640244 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 29/60] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:17:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:08.645617 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e176ac9-5fe2-47be-839b-1efff16560c4 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:08.695731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:08.741318 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e176ac9-5fe2-47be-839b-1efff16560c4 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:08.772934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2fc92-6ae4-4d8b-adaa-59dcbaf77cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:08.773539 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2fc92-6ae4-4d8b-adaa-59dcbaf77cda None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:08.773820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2fc92-6ae4-4d8b-adaa-59dcbaf77cda None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:08.774095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2fc92-6ae4-4d8b-adaa-59dcbaf77cda None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:08.774768 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2fc92-6ae4-4d8b-adaa-59dcbaf77cda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:08.776767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2fc92-6ae4-4d8b-adaa-59dcbaf77cda None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:08.801980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 32/61] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Aug 6 12:17:08 2022] POST /identity/v3/users => generated 275 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:17:11.982958 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe80c5-1273-4e0a-9930-f9e99747692f None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:11.983461 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe80c5-1273-4e0a-9930-f9e99747692f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:11.983533 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe80c5-1273-4e0a-9930-f9e99747692f None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:11.985140 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 30/62] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:11.990969 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5f1b0-19dd-45c8-b6bf-835cfb88a0f1 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:11.991364 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5f1b0-19dd-45c8-b6bf-835cfb88a0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:11.991533 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5f1b0-19dd-45c8-b6bf-835cfb88a0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:11.993776 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0365b-9ddd-449e-86b1-47f1499bb854 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:11.994058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0365b-9ddd-449e-86b1-47f1499bb854 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:11.994331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0365b-9ddd-449e-86b1-47f1499bb854 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:11.996160 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 31/63] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:17:12.001323 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf869b-4997-4162-9850-fcfeb488e57c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:12.001881 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf869b-4997-4162-9850-fcfeb488e57c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:12.002376 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf869b-4997-4162-9850-fcfeb488e57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:12.021936 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-91d5f1b0-19dd-45c8-b6bf-835cfb88a0f1 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:12.023346 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-91d5f1b0-19dd-45c8-b6bf-835cfb88a0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:12.026088 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ceaf869b-4997-4162-9850-fcfeb488e57c None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:12.028677 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ceaf869b-4997-4162-9850-fcfeb488e57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:12.097043 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 33/64] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 12:17:12.098619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d0bea-f84a-4e78-978f-7f84ad775fac None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:12.099037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d0bea-f84a-4e78-978f-7f84ad775fac None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:12.099535 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d0bea-f84a-4e78-978f-7f84ad775fac None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:12.100284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:12.101804 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 34/65] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:12.114589 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059b60b-58b4-41fb-a38a-8fc736b7a7bd None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:12.115312 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059b60b-58b4-41fb-a38a-8fc736b7a7bd None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:12.115726 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059b60b-58b4-41fb-a38a-8fc736b7a7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:12.121262 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 32/66] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:12.128325 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf869b-4997-4162-9850-fcfeb488e57c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:12.149585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-f059b60b-58b4-41fb-a38a-8fc736b7a7bd None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:12.150926 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f059b60b-58b4-41fb-a38a-8fc736b7a7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:12.183050 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:12.230268 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 35/67] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:12.230919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f059b60b-58b4-41fb-a38a-8fc736b7a7bd None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:12.257055 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf869b-4997-4162-9850-fcfeb488e57c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:12.308759 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47bbc4f-60e2-4ff1-8c07-6dc949b4856f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:12.310233 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bbc4f-60e2-4ff1-8c07-6dc949b4856f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:12.310692 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bbc4f-60e2-4ff1-8c07-6dc949b4856f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:12.311415 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bbc4f-60e2-4ff1-8c07-6dc949b4856f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:12.328448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47bbc4f-60e2-4ff1-8c07-6dc949b4856f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:12.335063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-f47bbc4f-60e2-4ff1-8c07-6dc949b4856f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:17:12.335063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:12.335063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:12.335063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:12.335063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:12.335063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:12.335063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:12.335063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:12.335063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:12.335063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:12.335063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:12.335063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:12.335063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:12.335063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:12.335063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:12.335063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:12.336004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:12.336004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:12.336004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:12.336004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:12.336004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:12.336004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:12.336004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:12.336004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:12.336004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:12.336004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:12.336004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:12.336004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:12.336004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:12.336004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:12.336004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:12.336004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:12.336803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:12.336803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:12.336803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:12.336803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:12.336803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:12.336803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:12.336803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:12.336803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:12.336803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:12.336803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:12.336803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:17:12.336803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:12.340358 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 33/68] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Aug 6 12:17:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:17:12.340358 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47bbc4f-60e2-4ff1-8c07-6dc949b4856f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:12.366335 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f059b60b-58b4-41fb-a38a-8fc736b7a7bd None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:12.405205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c3604-18b2-4d79-9420-6550cdf58c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:12.405952 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c3604-18b2-4d79-9420-6550cdf58c44 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:12.406231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c3604-18b2-4d79-9420-6550cdf58c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:12.406509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c3604-18b2-4d79-9420-6550cdf58c44 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:12.417260 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c3604-18b2-4d79-9420-6550cdf58c44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:12.417719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:12.422369 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-6a1c3604-18b2-4d79-9420-6550cdf58c44 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:12.422369 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:12.422369 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:12.422369 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:12.422369 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:12.422369 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:12.422369 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:12.422369 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:12.422369 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:12.422369 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:12.422369 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:12.422369 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:12.422369 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:12.422369 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:12.422369 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:12.422369 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:12.423205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:12.423205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:12.423205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:12.423205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:12.423205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:12.423205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:12.423205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:12.423205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:12.423205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:12.423205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:12.423205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:12.423205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:12.423205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:12.423205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:12.423205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:12.423205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:12.423959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:12.423959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:12.423959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:12.423959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:12.423959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:12.423959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:12.423959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:12.423959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:12.423959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:12.423959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:12.423959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:12.423959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:17:12.423959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 36/69] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:12.424629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c3604-18b2-4d79-9420-6550cdf58c44 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:12.453192 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:12.466029 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47bbc4f-60e2-4ff1-8c07-6dc949b4856f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:12.503088 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b467ad-836d-4850-b7d8-a620b8cc6ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:12.504068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b467ad-836d-4850-b7d8-a620b8cc6ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:12.504564 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b467ad-836d-4850-b7d8-a620b8cc6ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:12.505005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b467ad-836d-4850-b7d8-a620b8cc6ec7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:12.515976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b467ad-836d-4850-b7d8-a620b8cc6ec7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:12.521237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-00b467ad-836d-4850-b7d8-a620b8cc6ec7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:17:12.521237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:12.521237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:12.521237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:12.521237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:12.521237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:12.521237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:12.521237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:12.521237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:12.521237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:12.521237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:12.521237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:12.521237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:12.521237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:12.521237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:12.521237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:12.522056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:12.522056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:12.522056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:12.522056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:12.522056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:12.522056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:12.522056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:12.522056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:12.522056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:12.522056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:12.522056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:12.522056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:12.522056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:12.522056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:12.522056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:12.522056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:12.522804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:12.522804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:12.522804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:12.522804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:12.522804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:12.522804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:12.522804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:12.522804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:12.522804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:12.522804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:12.522804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:17:12.522804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:12.524691 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 34/70] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:17:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:12.528463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7d8a3-44d8-486f-874a-370700bc4371 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:12.528463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7d8a3-44d8-486f-874a-370700bc4371 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:12.528463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7d8a3-44d8-486f-874a-370700bc4371 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:12.532052 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 35/71] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:12.532226 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c3604-18b2-4d79-9420-6550cdf58c44 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:12.532743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7d8a3-44d8-486f-874a-370700bc4371 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:12.567881 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e87e2-e5ad-4dc2-b50a-56fb1cacc169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:12.568528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e87e2-e5ad-4dc2-b50a-56fb1cacc169 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:12.568821 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e87e2-e5ad-4dc2-b50a-56fb1cacc169 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:12.569094 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e87e2-e5ad-4dc2-b50a-56fb1cacc169 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:12.569810 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e87e2-e5ad-4dc2-b50a-56fb1cacc169 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:12.570115 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e87e2-e5ad-4dc2-b50a-56fb1cacc169 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:12.571511 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e87e2-e5ad-4dc2-b50a-56fb1cacc169 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:12.580021 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 37/72] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:17:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:12.580839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e87e2-e5ad-4dc2-b50a-56fb1cacc169 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:12.640421 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7d8a3-44d8-486f-874a-370700bc4371 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:12.659638 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:12.670300 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:12.674384 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f86521-e857-4637-ae2a-8fdd6fc8d69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:12.674994 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f86521-e857-4637-ae2a-8fdd6fc8d69c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:12.675276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f86521-e857-4637-ae2a-8fdd6fc8d69c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:12.675551 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f86521-e857-4637-ae2a-8fdd6fc8d69c None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:12.676263 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f86521-e857-4637-ae2a-8fdd6fc8d69c None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:12.676599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f86521-e857-4637-ae2a-8fdd6fc8d69c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:12.677970 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f86521-e857-4637-ae2a-8fdd6fc8d69c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:12.680002 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e87e2-e5ad-4dc2-b50a-56fb1cacc169 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:12.687050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 36/73] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:12.688196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869f40c-5341-42b3-97cb-aaae9a16c238 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:12.688508 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869f40c-5341-42b3-97cb-aaae9a16c238 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:12.688783 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869f40c-5341-42b3-97cb-aaae9a16c238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:12.705691 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e869f40c-5341-42b3-97cb-aaae9a16c238 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:12.706872 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e869f40c-5341-42b3-97cb-aaae9a16c238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:12.713795 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd87cde-60b4-46c7-8b89-6782017d8f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:12.715189 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd87cde-60b4-46c7-8b89-6782017d8f1c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:12.715483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd87cde-60b4-46c7-8b89-6782017d8f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:12.715757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd87cde-60b4-46c7-8b89-6782017d8f1c None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:12.716503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd87cde-60b4-46c7-8b89-6782017d8f1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:12.716816 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd87cde-60b4-46c7-8b89-6782017d8f1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:12.719165 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd87cde-60b4-46c7-8b89-6782017d8f1c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:12.727155 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 38/74] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:12.728520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534fd7c-15f3-484f-8f9a-daabe29a6e5d None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:12.728812 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534fd7c-15f3-484f-8f9a-daabe29a6e5d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:12.729207 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534fd7c-15f3-484f-8f9a-daabe29a6e5d None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:12.730437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 39/75] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:12.734397 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534fd7c-15f3-484f-8f9a-daabe29a6e5d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:12.779045 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 37/76] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:12.779741 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869f40c-5341-42b3-97cb-aaae9a16c238 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:12.845237 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534fd7c-15f3-484f-8f9a-daabe29a6e5d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:12.876774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869f40c-5341-42b3-97cb-aaae9a16c238 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:12.878059 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d072e-a050-4e99-a6ea-112276753d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:12.878692 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d072e-a050-4e99-a6ea-112276753d4e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:12.878971 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d072e-a050-4e99-a6ea-112276753d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:12.879243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d072e-a050-4e99-a6ea-112276753d4e None admin] PATH_INFO: `/v3/users/91c44362773b4667825e3087cb73188e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:12.889508 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:12.892291 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d072e-a050-4e99-a6ea-112276753d4e None admin] RBAC: Authorizing `identity:get_user(user_id=91c44362773b4667825e3087cb73188e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:12.893768 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d072e-a050-4e99-a6ea-112276753d4e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:12.895398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 40/77] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:12 2022] GET /identity/v3/users/91c44362773b4667825e3087cb73188e => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:12.895398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d072e-a050-4e99-a6ea-112276753d4e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:12.907355 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:12.914452 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31888d5a-8bf6-4381-a992-58984461416e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:12.915099 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31888d5a-8bf6-4381-a992-58984461416e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:12.915381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31888d5a-8bf6-4381-a992-58984461416e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:12.915658 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31888d5a-8bf6-4381-a992-58984461416e None admin] PATH_INFO: `/v3/users/91c44362773b4667825e3087cb73188e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:12.917352 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31888d5a-8bf6-4381-a992-58984461416e None admin] RBAC: Authorizing `identity:get_user(user_id=91c44362773b4667825e3087cb73188e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:12.918791 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31888d5a-8bf6-4381-a992-58984461416e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:12.920580 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 38/78] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:12 2022] GET /identity/v3/users/91c44362773b4667825e3087cb73188e => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:12.921876 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6405db8-ae76-4d60-bb10-c3e2227bf92f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:12.922002 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6405db8-ae76-4d60-bb10-c3e2227bf92f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:12.922203 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6405db8-ae76-4d60-bb10-c3e2227bf92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:12.949005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b6405db8-ae76-4d60-bb10-c3e2227bf92f None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:12.951334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b6405db8-ae76-4d60-bb10-c3e2227bf92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:13.011264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d072e-a050-4e99-a6ea-112276753d4e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:13.029946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 39/79] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:13.030763 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6405db8-ae76-4d60-bb10-c3e2227bf92f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:13.059364 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c125d91-b335-46bc-8144-1e2cbc022f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:13.060011 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c125d91-b335-46bc-8144-1e2cbc022f9d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:13.060329 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c125d91-b335-46bc-8144-1e2cbc022f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:13.060597 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c125d91-b335-46bc-8144-1e2cbc022f9d None admin] PATH_INFO: `/v3/users/c9de148de73e459eb6456afdaf9fe213` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:13.062006 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c125d91-b335-46bc-8144-1e2cbc022f9d None admin] RBAC: Authorizing `identity:get_user(user_id=c9de148de73e459eb6456afdaf9fe213)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:13.063214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c125d91-b335-46bc-8144-1e2cbc022f9d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:13.068704 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 41/80] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:12 2022] GET /identity/v3/users/c9de148de73e459eb6456afdaf9fe213 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:13.068704 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c125d91-b335-46bc-8144-1e2cbc022f9d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:13.146556 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:13.162076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6405db8-ae76-4d60-bb10-c3e2227bf92f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:13.169431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c125d91-b335-46bc-8144-1e2cbc022f9d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:13.175450 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:13.200852 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4aea2e-104e-4bca-b8a3-e24cb2bb035e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:13.201488 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4aea2e-104e-4bca-b8a3-e24cb2bb035e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:13.201799 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4aea2e-104e-4bca-b8a3-e24cb2bb035e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:13.202076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4aea2e-104e-4bca-b8a3-e24cb2bb035e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:13.206778 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538625e-0ea7-4511-802c-6d25f49c8017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:13.207396 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538625e-0ea7-4511-802c-6d25f49c8017 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:13.207677 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538625e-0ea7-4511-802c-6d25f49c8017 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:13.207952 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538625e-0ea7-4511-802c-6d25f49c8017 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:13.212173 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4aea2e-104e-4bca-b8a3-e24cb2bb035e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:13.217713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-cf4aea2e-104e-4bca-b8a3-e24cb2bb035e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:13.217713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:13.217713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:13.217713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:13.217713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:13.217713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:13.217713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:13.217713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:13.217713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:13.217713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:13.217713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:13.217713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:13.217713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:13.217713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:13.217713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:13.217713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:13.218731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:13.218731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:13.218731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:13.218731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:13.218731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:13.218731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:13.218731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:13.218731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:13.218731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:13.218731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:13.218731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:13.218731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:13.218731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:13.218731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:13.218731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:13.218731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:13.219619 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:13.219619 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:13.219619 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:13.219619 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:13.219619 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:13.219619 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:13.219619 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:13.219619 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:13.219619 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:13.219619 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:13.219619 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:13.219619 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:13.224093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 40/81] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:13.224847 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4aea2e-104e-4bca-b8a3-e24cb2bb035e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:13.282658 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538625e-0ea7-4511-802c-6d25f49c8017 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:13.284096 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538625e-0ea7-4511-802c-6d25f49c8017 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:13.326361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4aea2e-104e-4bca-b8a3-e24cb2bb035e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:13.361082 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:13.361232 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39939f33-6824-4e2e-8855-5625b0b9481f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:13.361232 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39939f33-6824-4e2e-8855-5625b0b9481f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:13.361232 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39939f33-6824-4e2e-8855-5625b0b9481f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:13.361232 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39939f33-6824-4e2e-8855-5625b0b9481f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:13.380380 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 42/82] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:13.385198 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538625e-0ea7-4511-802c-6d25f49c8017 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:13.402888 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:13.431807 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39939f33-6824-4e2e-8855-5625b0b9481f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:13.433338 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39939f33-6824-4e2e-8855-5625b0b9481f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:13.477310 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538625e-0ea7-4511-802c-6d25f49c8017 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:13.512336 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084b52b-957d-495c-a9ff-65e520b71992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:13.512972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084b52b-957d-495c-a9ff-65e520b71992 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:13.513256 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084b52b-957d-495c-a9ff-65e520b71992 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:13.513536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2084b52b-957d-495c-a9ff-65e520b71992 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:13.514672 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2084b52b-957d-495c-a9ff-65e520b71992 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:13.516121 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2084b52b-957d-495c-a9ff-65e520b71992 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:13.517778 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 43/83] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:17:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:13.518542 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084b52b-957d-495c-a9ff-65e520b71992 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:13.531984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 41/84] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:13.532906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39939f33-6824-4e2e-8855-5625b0b9481f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:13.568836 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:13.598065 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:13.621421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2084b52b-957d-495c-a9ff-65e520b71992 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:13.628415 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39939f33-6824-4e2e-8855-5625b0b9481f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:13.654630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaff167-ca57-4af3-ac38-dc115ee938d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:13.655244 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaff167-ca57-4af3-ac38-dc115ee938d8 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:13.655530 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaff167-ca57-4af3-ac38-dc115ee938d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:13.655806 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaff167-ca57-4af3-ac38-dc115ee938d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:13.660758 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a71e3b-8c46-4784-9aea-16d68018591e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:13.661365 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a71e3b-8c46-4784-9aea-16d68018591e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:13.661644 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a71e3b-8c46-4784-9aea-16d68018591e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:13.661918 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a71e3b-8c46-4784-9aea-16d68018591e None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:13.662683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a71e3b-8c46-4784-9aea-16d68018591e None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:13.663004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a71e3b-8c46-4784-9aea-16d68018591e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:13.664374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a71e3b-8c46-4784-9aea-16d68018591e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:13.672895 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 42/85] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:13.672895 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a71e3b-8c46-4784-9aea-16d68018591e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:13.727187 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaff167-ca57-4af3-ac38-dc115ee938d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:13.728675 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaff167-ca57-4af3-ac38-dc115ee938d8 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:13.769195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a71e3b-8c46-4784-9aea-16d68018591e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:13.773015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:13.798087 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:13.803507 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f79cc-81e4-4a02-86d4-80eeba4bacdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:13.804141 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f79cc-81e4-4a02-86d4-80eeba4bacdf None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:13.804459 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f79cc-81e4-4a02-86d4-80eeba4bacdf None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:13.804732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f79cc-81e4-4a02-86d4-80eeba4bacdf None admin] PATH_INFO: `/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:13.805899 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f79cc-81e4-4a02-86d4-80eeba4bacdf None admin] RBAC: Authorizing `identity:get_project(project_id=5a84e3fab8334ecca0cd4aae83cb5317)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:13.807359 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f79cc-81e4-4a02-86d4-80eeba4bacdf None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:13.809026 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 43/86] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:13 2022] GET /identity/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317 => generated 298 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:13.813992 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f79cc-81e4-4a02-86d4-80eeba4bacdf None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:13.826558 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 44/87] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:13.827302 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaff167-ca57-4af3-ac38-dc115ee938d8 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:13.908520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f79cc-81e4-4a02-86d4-80eeba4bacdf None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:13.922915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaff167-ca57-4af3-ac38-dc115ee938d8 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:13.943505 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0c78a-70c2-41e9-83f7-c8ee38cae748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:13.944411 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0c78a-70c2-41e9-83f7-c8ee38cae748 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:13.944702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0c78a-70c2-41e9-83f7-c8ee38cae748 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:13.944985 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0c78a-70c2-41e9-83f7-c8ee38cae748 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:13.945678 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0c78a-70c2-41e9-83f7-c8ee38cae748 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:13.947124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0c78a-70c2-41e9-83f7-c8ee38cae748 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:13.958591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890ab35-341e-48d3-80c9-57bd17fcd577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:13.959232 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890ab35-341e-48d3-80c9-57bd17fcd577 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:13.959516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890ab35-341e-48d3-80c9-57bd17fcd577 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:13.959792 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890ab35-341e-48d3-80c9-57bd17fcd577 None admin] PATH_INFO: `/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:13.960958 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7890ab35-341e-48d3-80c9-57bd17fcd577 None admin] RBAC: Authorizing `identity:get_project(project_id=5a84e3fab8334ecca0cd4aae83cb5317)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:13.962412 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7890ab35-341e-48d3-80c9-57bd17fcd577 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:13.964065 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 45/88] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:13 2022] GET /identity/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317 => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:13.964923 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890ab35-341e-48d3-80c9-57bd17fcd577 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:13.992331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 44/89] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Aug 6 12:17:13 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:17:13.992331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0c78a-70c2-41e9-83f7-c8ee38cae748 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:13.995471 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:14.049412 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:14.098727 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0c78a-70c2-41e9-83f7-c8ee38cae748 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:14.100121 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890ab35-341e-48d3-80c9-57bd17fcd577 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:14.137681 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1405b-fa85-4a44-9165-389d80fe606e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:14.138754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1405b-fa85-4a44-9165-389d80fe606e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:14.138754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1405b-fa85-4a44-9165-389d80fe606e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:14.138915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1405b-fa85-4a44-9165-389d80fe606e None admin] PATH_INFO: `/v3/users/91c44362773b4667825e3087cb73188e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:14.140034 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa1405b-fa85-4a44-9165-389d80fe606e None admin] RBAC: Authorizing `identity:get_user(user_id=91c44362773b4667825e3087cb73188e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:14.141548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa1405b-fa85-4a44-9165-389d80fe606e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:14.142506 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454cded4-151b-4fc6-9c33-81dfd028e251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:14.143180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cded4-151b-4fc6-9c33-81dfd028e251 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:14.143380 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 46/90] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:13 2022] GET /identity/v3/users/91c44362773b4667825e3087cb73188e => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:14.143464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cded4-151b-4fc6-9c33-81dfd028e251 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:14.143691 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cded4-151b-4fc6-9c33-81dfd028e251 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:14.143899 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1405b-fa85-4a44-9165-389d80fe606e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:14.144533 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454cded4-151b-4fc6-9c33-81dfd028e251 None admin] RBAC: Adding query filter params (role.id=47cde553d7ee465bbe685a05c00b94e2, scope.project.id=5a84e3fab8334ecca0cd4aae83cb5317, user.id=91c44362773b4667825e3087cb73188e) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:14.144849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454cded4-151b-4fc6-9c33-81dfd028e251 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:14.146311 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454cded4-151b-4fc6-9c33-81dfd028e251 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:14.156139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 45/91] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:17:13 2022] GET /identity/v3/role_assignments?user.id=91c44362773b4667825e3087cb73188e&scope.project.id=5a84e3fab8334ecca0cd4aae83cb5317&role.id=47cde553d7ee465bbe685a05c00b94e2 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:14.159799 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454cded4-151b-4fc6-9c33-81dfd028e251 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:14.273649 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:14.301973 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1405b-fa85-4a44-9165-389d80fe606e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:14.303369 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454cded4-151b-4fc6-9c33-81dfd028e251 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:14.329488 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:14.350670 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894711b-8e87-4f17-a33b-51331a7f95c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:14.351320 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894711b-8e87-4f17-a33b-51331a7f95c6 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:14.351609 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894711b-8e87-4f17-a33b-51331a7f95c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:14.351871 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894711b-8e87-4f17-a33b-51331a7f95c6 None admin] PATH_INFO: `/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:14.352454 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90dffef-d470-4437-8d5a-a1cc5efa2fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:14.353050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90dffef-d470-4437-8d5a-a1cc5efa2fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:14.353434 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90dffef-d470-4437-8d5a-a1cc5efa2fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:14.353781 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90dffef-d470-4437-8d5a-a1cc5efa2fe6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:14.355124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90dffef-d470-4437-8d5a-a1cc5efa2fe6 None admin] RBAC: Adding query filter params (role.id=0fd7b75299b64c54971d9bfeaec30f4e, scope.project.id=5a84e3fab8334ecca0cd4aae83cb5317, user.id=91c44362773b4667825e3087cb73188e) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:14.355298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7894711b-8e87-4f17-a33b-51331a7f95c6 None admin] RBAC: Authorizing `identity:get_project(project_id=5a84e3fab8334ecca0cd4aae83cb5317)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:14.355456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90dffef-d470-4437-8d5a-a1cc5efa2fe6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:14.356820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7894711b-8e87-4f17-a33b-51331a7f95c6 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:14.357010 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90dffef-d470-4437-8d5a-a1cc5efa2fe6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:14.358527 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 47/92] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:14 2022] GET /identity/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:14.359307 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894711b-8e87-4f17-a33b-51331a7f95c6 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:14.366329 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 46/93] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:17:14 2022] GET /identity/v3/role_assignments?user.id=91c44362773b4667825e3087cb73188e&scope.project.id=5a84e3fab8334ecca0cd4aae83cb5317&role.id=0fd7b75299b64c54971d9bfeaec30f4e => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:14.367552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90dffef-d470-4437-8d5a-a1cc5efa2fe6 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:14.479669 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894711b-8e87-4f17-a33b-51331a7f95c6 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:14.511554 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90dffef-d470-4437-8d5a-a1cc5efa2fe6 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:14.528798 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae616f-9283-4b61-821f-30888adf23ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:14.529442 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae616f-9283-4b61-821f-30888adf23ee None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:14.529734 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae616f-9283-4b61-821f-30888adf23ee None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:14.530065 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae616f-9283-4b61-821f-30888adf23ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:14.551363 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff2495-5d17-4f7e-ab7d-944e036c34f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:14.552553 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff2495-5d17-4f7e-ab7d-944e036c34f0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:14.553089 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff2495-5d17-4f7e-ab7d-944e036c34f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:14.553826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff2495-5d17-4f7e-ab7d-944e036c34f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:14.554936 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff2495-5d17-4f7e-ab7d-944e036c34f0 None admin] RBAC: Adding query filter params (role.id=144c8425688b409384e1e8ed8279d88e, scope.project.id=5a84e3fab8334ecca0cd4aae83cb5317, user.id=c9de148de73e459eb6456afdaf9fe213) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:14.555428 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:14.556650 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff2495-5d17-4f7e-ab7d-944e036c34f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:14.558819 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff2495-5d17-4f7e-ab7d-944e036c34f0 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:14.566930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 47/94] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:17:14 2022] GET /identity/v3/role_assignments?user.id=c9de148de73e459eb6456afdaf9fe213&scope.project.id=5a84e3fab8334ecca0cd4aae83cb5317&role.id=144c8425688b409384e1e8ed8279d88e => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:14.617554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae616f-9283-4b61-821f-30888adf23ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:14.619049 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae616f-9283-4b61-821f-30888adf23ee None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:14.731686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 48/95] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:14.737868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff2495-5d17-4f7e-ab7d-944e036c34f0 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:14.794670 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:14.853834 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff2495-5d17-4f7e-ab7d-944e036c34f0 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:14.891661 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774bccc2-ae93-40a3-ab2f-1a03db2739f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:14.892332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bccc2-ae93-40a3-ab2f-1a03db2739f8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:14.892625 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bccc2-ae93-40a3-ab2f-1a03db2739f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:14.892907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bccc2-ae93-40a3-ab2f-1a03db2739f8 None admin] PATH_INFO: `/v3/projects/5a58ca6c3969438e897eb2906823365f` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:14.894094 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774bccc2-ae93-40a3-ab2f-1a03db2739f8 None admin] RBAC: Authorizing `identity:get_project(project_id=5a58ca6c3969438e897eb2906823365f)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:14.895546 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774bccc2-ae93-40a3-ab2f-1a03db2739f8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:14.897653 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 48/96] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:14 2022] GET /identity/v3/projects/5a58ca6c3969438e897eb2906823365f => generated 312 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:14.904030 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae616f-9283-4b61-821f-30888adf23ee None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:14.964909 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:15.039274 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae616f-9283-4b61-821f-30888adf23ee None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:15.087825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d8fc8-1412-4f0c-9261-ccd76298d047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:15.088545 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d8fc8-1412-4f0c-9261-ccd76298d047 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:15.088832 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d8fc8-1412-4f0c-9261-ccd76298d047 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:15.089112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d8fc8-1412-4f0c-9261-ccd76298d047 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:15.089922 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d8fc8-1412-4f0c-9261-ccd76298d047 None admin] RBAC: Adding query filter params (role.id=0fd7b75299b64c54971d9bfeaec30f4e, scope.project.id=5a58ca6c3969438e897eb2906823365f, user.id=91c44362773b4667825e3087cb73188e) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:15.090221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d8fc8-1412-4f0c-9261-ccd76298d047 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:15.091764 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d8fc8-1412-4f0c-9261-ccd76298d047 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:15.100103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 49/97] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:17:14 2022] GET /identity/v3/role_assignments?user.id=91c44362773b4667825e3087cb73188e&scope.project.id=5a58ca6c3969438e897eb2906823365f&role.id=0fd7b75299b64c54971d9bfeaec30f4e => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:16.834097 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a271f-693a-4bee-ac5f-1144315163b0 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:16.834603 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a271f-693a-4bee-ac5f-1144315163b0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:16.834712 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a271f-693a-4bee-ac5f-1144315163b0 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:16.836387 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 49/98] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:16.841342 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27526f-e141-479f-a532-5c2dd13ca733 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:16.842154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27526f-e141-479f-a532-5c2dd13ca733 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:16.842663 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27526f-e141-479f-a532-5c2dd13ca733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:16.864887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ab27526f-e141-479f-a532-5c2dd13ca733 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:16.866821 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ab27526f-e141-479f-a532-5c2dd13ca733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:16.891673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbc64c-0788-423d-9931-b14e00be789c None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:16.892271 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbc64c-0788-423d-9931-b14e00be789c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:16.892776 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbc64c-0788-423d-9931-b14e00be789c None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:16.894594 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 50/99] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:16.901816 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aae5ea-cfba-496f-aa43-a7dcb5fe0800 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:16.902362 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aae5ea-cfba-496f-aa43-a7dcb5fe0800 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:16.902858 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aae5ea-cfba-496f-aa43-a7dcb5fe0800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:16.928537 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-61aae5ea-cfba-496f-aa43-a7dcb5fe0800 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:16.930776 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-61aae5ea-cfba-496f-aa43-a7dcb5fe0800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:17.008328 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 50/100] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:17.036570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27526f-e141-479f-a532-5c2dd13ca733 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:17.055358 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 51/101] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:17.089891 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aae5ea-cfba-496f-aa43-a7dcb5fe0800 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:17.111586 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:17.195832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:17.243911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27526f-e141-479f-a532-5c2dd13ca733 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:17.303872 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4e84d-81ee-471f-829e-b42741b05614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:17.304550 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4e84d-81ee-471f-829e-b42741b05614 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:17.304835 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4e84d-81ee-471f-829e-b42741b05614 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:17.305112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4e84d-81ee-471f-829e-b42741b05614 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:17.326501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aae5ea-cfba-496f-aa43-a7dcb5fe0800 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:17.329266 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4e84d-81ee-471f-829e-b42741b05614 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:17.335559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-70b4e84d-81ee-471f-829e-b42741b05614 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:17:17.335559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:17.335559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:17.335559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:17.335559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:17.335559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:17.335559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:17.335559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:17.335559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:17.335559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:17.335559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:17.335559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:17.335559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:17.335559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:17.335559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:17.335559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:17.336491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:17.336491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:17.336491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:17.336491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:17.336491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:17.336491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:17.336491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:17.336491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:17.336491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:17.336491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:17.336491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:17.336491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:17.336491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:17.336491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:17.336491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:17.336491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:17.337350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:17.337350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:17.337350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:17.337350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:17.337350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:17.337350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:17.337350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:17.337350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:17.337350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:17.337350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:17.337350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:17:17.337350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:17:17.337350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 51/102] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Aug 6 12:17:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:17:17.342733 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4e84d-81ee-471f-829e-b42741b05614 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:17.409327 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49779f45-3f0b-440c-92ea-a6c33ee60c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:17.409975 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49779f45-3f0b-440c-92ea-a6c33ee60c35 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:17.410255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49779f45-3f0b-440c-92ea-a6c33ee60c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:17.410529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49779f45-3f0b-440c-92ea-a6c33ee60c35 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:17.430285 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49779f45-3f0b-440c-92ea-a6c33ee60c35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:17.435448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-49779f45-3f0b-440c-92ea-a6c33ee60c35 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 12:17:17.435448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:17.435448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:17.435448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:17.435448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:17.435448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:17.435448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:17.435448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:17.435448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:17.435448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:17.435448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:17.435448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:17.435448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:17.435448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:17.435448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:17.435448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:17.436354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:17.436354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:17.436354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:17.436354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:17.436354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:17.436354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:17.436354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:17.436354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:17.436354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:17.436354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:17.436354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:17.436354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:17.436354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:17.436354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:17.436354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:17.436354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:17.437124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:17.437124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:17.437124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:17.437124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:17.437124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:17.437124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:17.437124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:17.437124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:17.437124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:17.437124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:17.437124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 12:17:17.437124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:17.440076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 52/103] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:17.445452 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aea8ad-612e-4cd0-a5eb-042f794e6d61 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:17.445742 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aea8ad-612e-4cd0-a5eb-042f794e6d61 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:17.446015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aea8ad-612e-4cd0-a5eb-042f794e6d61 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:17.448472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 53/104] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:17.449364 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aea8ad-612e-4cd0-a5eb-042f794e6d61 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:17.539299 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4e84d-81ee-471f-829e-b42741b05614 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:17.543168 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:17.590635 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b175d-4bd9-458c-9352-df70bc2375cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:17.591554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b175d-4bd9-458c-9352-df70bc2375cf None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:17.591984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b175d-4bd9-458c-9352-df70bc2375cf None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:17.592440 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b175d-4bd9-458c-9352-df70bc2375cf None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:17.593625 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b175d-4bd9-458c-9352-df70bc2375cf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:17.594049 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b175d-4bd9-458c-9352-df70bc2375cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:17.596263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b175d-4bd9-458c-9352-df70bc2375cf None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:17.609443 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aea8ad-612e-4cd0-a5eb-042f794e6d61 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:17.609752 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 52/105] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:17:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:17.611630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06109f53-1ed5-4ba0-9dfa-cbfc769bfcae None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:17.612235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06109f53-1ed5-4ba0-9dfa-cbfc769bfcae None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:17.612693 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06109f53-1ed5-4ba0-9dfa-cbfc769bfcae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:17.623088 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:17.636632 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-06109f53-1ed5-4ba0-9dfa-cbfc769bfcae None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:17.637858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-06109f53-1ed5-4ba0-9dfa-cbfc769bfcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:17.653336 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c90eb-b7d0-4d6f-bc1a-38d0ce4e149c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:17.654168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c90eb-b7d0-4d6f-bc1a-38d0ce4e149c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:17.654616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c90eb-b7d0-4d6f-bc1a-38d0ce4e149c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:17.655047 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c90eb-b7d0-4d6f-bc1a-38d0ce4e149c None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:17.655904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c90eb-b7d0-4d6f-bc1a-38d0ce4e149c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:17.656385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c90eb-b7d0-4d6f-bc1a-38d0ce4e149c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:17.657904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c90eb-b7d0-4d6f-bc1a-38d0ce4e149c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:17.666730 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 54/106] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:17.668319 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102a121-877a-4b99-b4bf-6b84897361bb None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:17.668843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102a121-877a-4b99-b4bf-6b84897361bb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:17.669312 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102a121-877a-4b99-b4bf-6b84897361bb None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:17.670980 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 55/107] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:17.671875 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102a121-877a-4b99-b4bf-6b84897361bb None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:17.727846 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 53/108] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:17.734002 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1f28c-474c-4c22-a808-ca6a1befe6c1 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:17.734002 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1f28c-474c-4c22-a808-ca6a1befe6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:17.734002 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1f28c-474c-4c22-a808-ca6a1befe6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:17.749233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-8ac1f28c-474c-4c22-a808-ca6a1befe6c1 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:17.750619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8ac1f28c-474c-4c22-a808-ca6a1befe6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:17.790533 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102a121-877a-4b99-b4bf-6b84897361bb None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:17.822457 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 54/109] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:17.823436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1f28c-474c-4c22-a808-ca6a1befe6c1 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:17.829947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd6eb0f-9a40-4581-9e19-92232dbe378b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:17.830857 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6eb0f-9a40-4581-9e19-92232dbe378b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:17.831454 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6eb0f-9a40-4581-9e19-92232dbe378b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:17.831739 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6eb0f-9a40-4581-9e19-92232dbe378b None admin] PATH_INFO: `/v3/users/91c44362773b4667825e3087cb73188e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:17.833303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd6eb0f-9a40-4581-9e19-92232dbe378b None admin] RBAC: Authorizing `identity:get_user(user_id=91c44362773b4667825e3087cb73188e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:17.835210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd6eb0f-9a40-4581-9e19-92232dbe378b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:17.836863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 56/110] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:17 2022] GET /identity/v3/users/91c44362773b4667825e3087cb73188e => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:17.841688 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f4051-d08b-4c1a-b84d-845d2fbf94ad None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:17.842199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f4051-d08b-4c1a-b84d-845d2fbf94ad None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:17.842473 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f4051-d08b-4c1a-b84d-845d2fbf94ad None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:17.843943 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 57/111] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:17.845453 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f4051-d08b-4c1a-b84d-845d2fbf94ad None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:17.859968 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:17.889061 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:17.943395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1f28c-474c-4c22-a808-ca6a1befe6c1 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:17.959429 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f4051-d08b-4c1a-b84d-845d2fbf94ad None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:17.983122 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16580e-ab44-493b-8fa4-561162aeacda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:17.983877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16580e-ab44-493b-8fa4-561162aeacda None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:17.984161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16580e-ab44-493b-8fa4-561162aeacda None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:17.984464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16580e-ab44-493b-8fa4-561162aeacda None admin] PATH_INFO: `/v3/users/f8d8e060c5cd456fa2c4a58592080111` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:18.001682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee39ea-326d-4920-bc20-98a1aead8375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:18.002351 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee39ea-326d-4920-bc20-98a1aead8375 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:18.002641 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee39ea-326d-4920-bc20-98a1aead8375 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:18.002730 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba16580e-ab44-493b-8fa4-561162aeacda None admin] RBAC: Authorizing `identity:get_user(user_id=f8d8e060c5cd456fa2c4a58592080111)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:18.002921 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee39ea-326d-4920-bc20-98a1aead8375 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:18.004280 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba16580e-ab44-493b-8fa4-561162aeacda None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:18.006226 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 55/112] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:17 2022] GET /identity/v3/users/f8d8e060c5cd456fa2c4a58592080111 => generated 289 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:18.007037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16580e-ab44-493b-8fa4-561162aeacda None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:18.014514 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee39ea-326d-4920-bc20-98a1aead8375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:18.019976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-c3ee39ea-326d-4920-bc20-98a1aead8375 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:18.019976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:18.019976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:18.019976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:18.019976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:18.019976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:18.019976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:18.019976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:18.019976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:18.019976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:18.019976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:18.019976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:18.019976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:18.019976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:18.019976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:18.019976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:18.020996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:18.020996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:18.020996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:18.020996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:18.020996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:18.020996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:18.020996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:18.020996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:18.020996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:18.020996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:18.020996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:18.020996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:18.020996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:18.020996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:18.020996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:18.020996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:18.021761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:18.021761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:18.021761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:18.021761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:18.021761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:18.021761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:18.021761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:18.021761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:18.021761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:18.021761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:18.021761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:18.021761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:18.021761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 58/113] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:18.022882 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f86d9-48fd-415c-9ca7-47e0787fbef2 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:18.022882 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f86d9-48fd-415c-9ca7-47e0787fbef2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:18.022882 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f86d9-48fd-415c-9ca7-47e0787fbef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:18.040585 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4d5f86d9-48fd-415c-9ca7-47e0787fbef2 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:18.042323 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4d5f86d9-48fd-415c-9ca7-47e0787fbef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:18.100141 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:18.114716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 59/114] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:18.117174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f86d9-48fd-415c-9ca7-47e0787fbef2 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:18.119345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16580e-ab44-493b-8fa4-561162aeacda None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:18.149445 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:18.158263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f49150-e388-41f2-8976-da765d85fac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:18.159694 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f49150-e388-41f2-8976-da765d85fac7 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:18.159978 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f49150-e388-41f2-8976-da765d85fac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:18.160254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f49150-e388-41f2-8976-da765d85fac7 None admin] PATH_INFO: `/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:18.161458 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f49150-e388-41f2-8976-da765d85fac7 None admin] RBAC: Authorizing `identity:get_project(project_id=5a84e3fab8334ecca0cd4aae83cb5317)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:18.162989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f49150-e388-41f2-8976-da765d85fac7 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:18.170474 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 56/115] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:18 2022] GET /identity/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:18.170474 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f49150-e388-41f2-8976-da765d85fac7 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:18.231352 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f86d9-48fd-415c-9ca7-47e0787fbef2 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:18.269980 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3cf40-d221-4f5c-862b-7bd9ab606182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:18.270785 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3cf40-d221-4f5c-862b-7bd9ab606182 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:18.271385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3cf40-d221-4f5c-862b-7bd9ab606182 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:18.271670 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3cf40-d221-4f5c-862b-7bd9ab606182 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:18.276564 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f49150-e388-41f2-8976-da765d85fac7 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:18.284369 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec3cf40-d221-4f5c-862b-7bd9ab606182 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:18.290901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-eec3cf40-d221-4f5c-862b-7bd9ab606182 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:17:18.290901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:18.290901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:18.290901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:18.290901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:18.290901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:18.290901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:18.290901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:18.290901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:18.290901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:18.290901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:18.290901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:18.290901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:18.290901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:18.290901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:18.290901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:18.292331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:18.292331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:18.292331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:18.292331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:18.292331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:18.292331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:18.292331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:18.292331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:18.292331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:18.292331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:18.292331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:18.292331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:18.292331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:18.292331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:18.292331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:18.292331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:18.293124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:18.293124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:18.293124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:18.293124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:18.293124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:18.293124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:18.293124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:18.293124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:18.293124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:18.293124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:18.293124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:17:18.293124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:18.293124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 60/116] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:17:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:18.300944 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3cf40-d221-4f5c-862b-7bd9ab606182 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:18.310673 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b4d77-cf02-4350-81ea-95d3418dc7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:18.311290 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b4d77-cf02-4350-81ea-95d3418dc7ed None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:18.311569 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b4d77-cf02-4350-81ea-95d3418dc7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:18.311840 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b4d77-cf02-4350-81ea-95d3418dc7ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:18.333357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:18.382140 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b4d77-cf02-4350-81ea-95d3418dc7ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:18.383601 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b4d77-cf02-4350-81ea-95d3418dc7ed None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:18.400559 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:18.417031 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3cf40-d221-4f5c-862b-7bd9ab606182 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:18.450893 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4c729-f863-4128-8625-903f0ab822a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:18.451557 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4c729-f863-4128-8625-903f0ab822a4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:18.451843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4c729-f863-4128-8625-903f0ab822a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:18.452121 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4c729-f863-4128-8625-903f0ab822a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:18.452874 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb4c729-f863-4128-8625-903f0ab822a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:18.453177 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb4c729-f863-4128-8625-903f0ab822a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:18.454564 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb4c729-f863-4128-8625-903f0ab822a4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:18.462463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 61/117] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:18.463243 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4c729-f863-4128-8625-903f0ab822a4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:18.483626 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 57/118] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:18.484564 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b4d77-cf02-4350-81ea-95d3418dc7ed None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:18.535297 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:18.568910 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4c729-f863-4128-8625-903f0ab822a4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:18.581656 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b4d77-cf02-4350-81ea-95d3418dc7ed None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:18.602936 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd16ca6e-70ac-417a-bf67-05e5a90c900e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:18.603574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16ca6e-70ac-417a-bf67-05e5a90c900e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:18.603881 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16ca6e-70ac-417a-bf67-05e5a90c900e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:18.604197 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16ca6e-70ac-417a-bf67-05e5a90c900e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:18.605457 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd16ca6e-70ac-417a-bf67-05e5a90c900e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:18.606921 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd16ca6e-70ac-417a-bf67-05e5a90c900e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:18.608550 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 62/119] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:17:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:18.609355 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd16ca6e-70ac-417a-bf67-05e5a90c900e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:18.616792 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77e2d0-d809-4cdc-89da-6a4f0b1001d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:18.617456 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77e2d0-d809-4cdc-89da-6a4f0b1001d0 None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:18.617737 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77e2d0-d809-4cdc-89da-6a4f0b1001d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:18.618014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77e2d0-d809-4cdc-89da-6a4f0b1001d0 None admin] PATH_INFO: `/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317/users/91c44362773b4667825e3087cb73188e/roles/47cde553d7ee465bbe685a05c00b94e2` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:18.620729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b77e2d0-d809-4cdc-89da-6a4f0b1001d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a84e3fab8334ecca0cd4aae83cb5317, user_id=91c44362773b4667825e3087cb73188e, role_id=47cde553d7ee465bbe685a05c00b94e2)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:18.622198 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b77e2d0-d809-4cdc-89da-6a4f0b1001d0 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:18.627587 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:18.634311 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 58/120] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:17:18 2022] PUT /identity/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317/users/91c44362773b4667825e3087cb73188e/roles/47cde553d7ee465bbe685a05c00b94e2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:17:18.635506 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ec9a5-7424-4b73-8c05-9cbd7babb97a None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:18.635796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ec9a5-7424-4b73-8c05-9cbd7babb97a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:18.636076 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ec9a5-7424-4b73-8c05-9cbd7babb97a None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:18.637371 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 59/121] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:18.638095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ec9a5-7424-4b73-8c05-9cbd7babb97a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:18.729397 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd16ca6e-70ac-417a-bf67-05e5a90c900e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:18.759541 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ec9a5-7424-4b73-8c05-9cbd7babb97a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:18.771069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959a999-24c0-49eb-8c31-51123da4ede4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:18.771737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959a999-24c0-49eb-8c31-51123da4ede4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:18.772073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959a999-24c0-49eb-8c31-51123da4ede4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:18.772396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959a999-24c0-49eb-8c31-51123da4ede4 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:18.773146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959a999-24c0-49eb-8c31-51123da4ede4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:18.773458 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959a999-24c0-49eb-8c31-51123da4ede4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:18.774965 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959a999-24c0-49eb-8c31-51123da4ede4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:18.776210 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:18.789792 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 63/122] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:18.790919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959a999-24c0-49eb-8c31-51123da4ede4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:18.808194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64196cc-c95c-432b-902e-5bdde16858e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:18.808899 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64196cc-c95c-432b-902e-5bdde16858e5 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:18.809201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64196cc-c95c-432b-902e-5bdde16858e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:18.809469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64196cc-c95c-432b-902e-5bdde16858e5 None admin] PATH_INFO: `/v3/users/91c44362773b4667825e3087cb73188e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:18.810649 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64196cc-c95c-432b-902e-5bdde16858e5 None admin] RBAC: Authorizing `identity:get_user(user_id=91c44362773b4667825e3087cb73188e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:18.812127 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64196cc-c95c-432b-902e-5bdde16858e5 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:18.813966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 60/123] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:18 2022] GET /identity/v3/users/91c44362773b4667825e3087cb73188e => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:18.814544 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64196cc-c95c-432b-902e-5bdde16858e5 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:18.882885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:18.908139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959a999-24c0-49eb-8c31-51123da4ede4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:18.920010 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64196cc-c95c-432b-902e-5bdde16858e5 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:18.944153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f71904-de57-423d-a98b-3c68f9d93d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:18.944835 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f71904-de57-423d-a98b-3c68f9d93d79 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:18.945121 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f71904-de57-423d-a98b-3c68f9d93d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:18.945401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f71904-de57-423d-a98b-3c68f9d93d79 None admin] PATH_INFO: `/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:18.946577 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f71904-de57-423d-a98b-3c68f9d93d79 None admin] RBAC: Authorizing `identity:get_project(project_id=5a84e3fab8334ecca0cd4aae83cb5317)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:18.948615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f71904-de57-423d-a98b-3c68f9d93d79 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:18.950292 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 64/124] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:18 2022] GET /identity/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:18.951420 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697fb86-7157-4786-b64e-b06f51d31920 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:18.951707 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697fb86-7157-4786-b64e-b06f51d31920 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:18.951985 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697fb86-7157-4786-b64e-b06f51d31920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:18.959639 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c1034-97f6-4cd4-9e83-93fa7d21bc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:18.960292 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c1034-97f6-4cd4-9e83-93fa7d21bc39 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:18.960604 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c1034-97f6-4cd4-9e83-93fa7d21bc39 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:18.960891 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2c1034-97f6-4cd4-9e83-93fa7d21bc39 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:18.961563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c1034-97f6-4cd4-9e83-93fa7d21bc39 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:18.963028 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2c1034-97f6-4cd4-9e83-93fa7d21bc39 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:18.970074 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e697fb86-7157-4786-b64e-b06f51d31920 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:18.971263 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e697fb86-7157-4786-b64e-b06f51d31920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:18.990620 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 61/125] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Aug 6 12:17:18 2022] POST /identity/v3/projects => generated 302 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:17:18.991269 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c1034-97f6-4cd4-9e83-93fa7d21bc39 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:19.042119 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:19.082240 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 65/126] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:19.084395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697fb86-7157-4786-b64e-b06f51d31920 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:19.139082 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2c1034-97f6-4cd4-9e83-93fa7d21bc39 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:19.179602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae914b-9994-4b54-854f-e25cab4ef354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:19.180224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae914b-9994-4b54-854f-e25cab4ef354 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:19.180545 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae914b-9994-4b54-854f-e25cab4ef354 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:19.180825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae914b-9994-4b54-854f-e25cab4ef354 None admin] PATH_INFO: `/v3/users/c9de148de73e459eb6456afdaf9fe213` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:19.182555 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ae914b-9994-4b54-854f-e25cab4ef354 None admin] RBAC: Authorizing `identity:get_user(user_id=c9de148de73e459eb6456afdaf9fe213)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:19.184942 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ae914b-9994-4b54-854f-e25cab4ef354 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:19.187833 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 62/127] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:18 2022] GET /identity/v3/users/c9de148de73e459eb6456afdaf9fe213 => generated 247 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:19.188565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:19.189655 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae914b-9994-4b54-854f-e25cab4ef354 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:19.218490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697fb86-7157-4786-b64e-b06f51d31920 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:19.263178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f43851-daf5-486d-9d45-6c509330364f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:19.264168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f43851-daf5-486d-9d45-6c509330364f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:19.264717 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f43851-daf5-486d-9d45-6c509330364f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:19.265178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f43851-daf5-486d-9d45-6c509330364f None admin] PATH_INFO: `/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:19.266556 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f43851-daf5-486d-9d45-6c509330364f None admin] RBAC: Authorizing `identity:get_project(project_id=5a84e3fab8334ecca0cd4aae83cb5317)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:19.268200 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f43851-daf5-486d-9d45-6c509330364f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:19.270114 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 66/128] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:19 2022] GET /identity/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:19.271147 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f43851-daf5-486d-9d45-6c509330364f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:19.313089 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:19.316337 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae914b-9994-4b54-854f-e25cab4ef354 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:19.355031 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1de07-2539-4ba4-949e-6566cf025ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:19.355756 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1de07-2539-4ba4-949e-6566cf025ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:19.356046 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1de07-2539-4ba4-949e-6566cf025ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:19.356447 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1de07-2539-4ba4-949e-6566cf025ac4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:19.357472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1de07-2539-4ba4-949e-6566cf025ac4 None admin] RBAC: Adding query filter params (role.id=7f46f07e73294f919cb56f7a97b324d7, scope.project.id=5a84e3fab8334ecca0cd4aae83cb5317, user.id=f8d8e060c5cd456fa2c4a58592080111) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:19.357792 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1de07-2539-4ba4-949e-6566cf025ac4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:19.359501 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1de07-2539-4ba4-949e-6566cf025ac4 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:19.365759 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 63/129] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:17:19 2022] GET /identity/v3/role_assignments?user.id=f8d8e060c5cd456fa2c4a58592080111&scope.project.id=5a84e3fab8334ecca0cd4aae83cb5317&role.id=7f46f07e73294f919cb56f7a97b324d7 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:19.372929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1de07-2539-4ba4-949e-6566cf025ac4 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:19.398565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f43851-daf5-486d-9d45-6c509330364f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:19.446017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542b9c3-93ce-407d-8b07-ed21076ec8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:19.448017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542b9c3-93ce-407d-8b07-ed21076ec8af None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:19.448558 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542b9c3-93ce-407d-8b07-ed21076ec8af None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:19.449030 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542b9c3-93ce-407d-8b07-ed21076ec8af None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:19.473374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542b9c3-93ce-407d-8b07-ed21076ec8af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:19.481069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-b542b9c3-93ce-407d-8b07-ed21076ec8af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:19.481069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:19.481069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:19.481069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:19.481069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:19.481069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:19.481069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:19.481069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:19.481069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:19.481069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:19.481069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:19.481069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:19.481069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:19.481069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:19.481069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:19.481069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:19.481977 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:19.481977 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:19.481977 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:19.481977 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:19.481977 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:19.481977 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:19.481977 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:19.481977 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:19.481977 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:19.481977 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:19.481977 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:19.481977 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:19.481977 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:19.481977 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:19.481977 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:19.481977 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:19.482811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:19.482811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:19.482811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:19.482811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:19.482811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:19.482811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:19.482811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:19.482811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:19.482811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:19.482811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:19.482811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:19.482811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:19.490572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 67/130] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:19.492408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542b9c3-93ce-407d-8b07-ed21076ec8af None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:19.502464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:19.589726 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1de07-2539-4ba4-949e-6566cf025ac4 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:19.662283 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd9233-e098-4045-87a7-1ea586069340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:19.663274 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd9233-e098-4045-87a7-1ea586069340 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:19.663749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd9233-e098-4045-87a7-1ea586069340 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:19.664830 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd9233-e098-4045-87a7-1ea586069340 None admin] PATH_INFO: `/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:19.667956 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cd9233-e098-4045-87a7-1ea586069340 None admin] RBAC: Authorizing `identity:get_project(project_id=5a84e3fab8334ecca0cd4aae83cb5317)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:19.669650 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cd9233-e098-4045-87a7-1ea586069340 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:19.673589 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 64/131] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:19 2022] GET /identity/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317 => generated 298 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:19.676573 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd9233-e098-4045-87a7-1ea586069340 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:19.703735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542b9c3-93ce-407d-8b07-ed21076ec8af None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:19.744512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:19.779749 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2cc49a-c4ac-45e4-a764-1a5755d434f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:19.782244 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2cc49a-c4ac-45e4-a764-1a5755d434f8 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:19.783657 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2cc49a-c4ac-45e4-a764-1a5755d434f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:19.784539 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2cc49a-c4ac-45e4-a764-1a5755d434f8 None admin] PATH_INFO: `/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317/users/91c44362773b4667825e3087cb73188e/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:19.788318 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2cc49a-c4ac-45e4-a764-1a5755d434f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a84e3fab8334ecca0cd4aae83cb5317, user_id=91c44362773b4667825e3087cb73188e, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:19.791094 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2cc49a-c4ac-45e4-a764-1a5755d434f8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:19.820358 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 68/132] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:17:19 2022] PUT /identity/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317/users/91c44362773b4667825e3087cb73188e/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:17:19.825341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2cc49a-c4ac-45e4-a764-1a5755d434f8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:19.918752 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd9233-e098-4045-87a7-1ea586069340 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:19.920787 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:19.964939 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2cc49a-c4ac-45e4-a764-1a5755d434f8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:19.993053 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411cb04c-a5bb-40dc-9f20-58427aba416e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:19.993736 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411cb04c-a5bb-40dc-9f20-58427aba416e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:19.993971 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411cb04c-a5bb-40dc-9f20-58427aba416e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:19.994256 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411cb04c-a5bb-40dc-9f20-58427aba416e None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:19.997622 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411cb04c-a5bb-40dc-9f20-58427aba416e None admin] RBAC: Adding query filter params (name=member) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:19.997622 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411cb04c-a5bb-40dc-9f20-58427aba416e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:19.997622 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411cb04c-a5bb-40dc-9f20-58427aba416e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:20.016364 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 65/133] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:20.022235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411cb04c-a5bb-40dc-9f20-58427aba416e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:20.033043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea4461-28a4-40a8-a300-293ed7274409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:20.034189 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea4461-28a4-40a8-a300-293ed7274409 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:20.034664 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea4461-28a4-40a8-a300-293ed7274409 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:20.035146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea4461-28a4-40a8-a300-293ed7274409 None admin] PATH_INFO: `/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317/users/c9de148de73e459eb6456afdaf9fe213/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:20.037894 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea4461-28a4-40a8-a300-293ed7274409 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a84e3fab8334ecca0cd4aae83cb5317, user_id=c9de148de73e459eb6456afdaf9fe213, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:20.039927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea4461-28a4-40a8-a300-293ed7274409 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:20.070943 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 69/134] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:17:19 2022] PUT /identity/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317/users/c9de148de73e459eb6456afdaf9fe213/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:17:20.164557 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:20.172324 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411cb04c-a5bb-40dc-9f20-58427aba416e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:20.216345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722eac77-76e4-405d-9b80-bc4e0c051b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:20.216913 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722eac77-76e4-405d-9b80-bc4e0c051b23 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:20.217197 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722eac77-76e4-405d-9b80-bc4e0c051b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:20.217476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722eac77-76e4-405d-9b80-bc4e0c051b23 None admin] PATH_INFO: `/v3/users/91c44362773b4667825e3087cb73188e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:20.220702 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722eac77-76e4-405d-9b80-bc4e0c051b23 None admin] RBAC: Authorizing `identity:get_user(user_id=91c44362773b4667825e3087cb73188e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:20.222194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722eac77-76e4-405d-9b80-bc4e0c051b23 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:20.223836 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 66/135] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:20 2022] GET /identity/v3/users/91c44362773b4667825e3087cb73188e => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:20.229473 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea4461-28a4-40a8-a300-293ed7274409 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:20.366372 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea4461-28a4-40a8-a300-293ed7274409 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:20.391113 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:20.403004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab38664-ca01-4b0a-85dd-ad568b869c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:20.403807 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab38664-ca01-4b0a-85dd-ad568b869c18 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:20.404330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab38664-ca01-4b0a-85dd-ad568b869c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:20.404706 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab38664-ca01-4b0a-85dd-ad568b869c18 None admin] PATH_INFO: `/v3/projects/5a58ca6c3969438e897eb2906823365f` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:20.406263 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab38664-ca01-4b0a-85dd-ad568b869c18 None admin] RBAC: Authorizing `identity:get_project(project_id=5a58ca6c3969438e897eb2906823365f)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:20.408270 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab38664-ca01-4b0a-85dd-ad568b869c18 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:20.410580 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 70/136] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:20 2022] GET /identity/v3/projects/5a58ca6c3969438e897eb2906823365f => generated 312 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:20.417551 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722eac77-76e4-405d-9b80-bc4e0c051b23 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:20.540395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722eac77-76e4-405d-9b80-bc4e0c051b23 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:20.575941 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc1d62-8c98-4212-b1e5-80591bc6b117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:20.576807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc1d62-8c98-4212-b1e5-80591bc6b117 None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:20.577187 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc1d62-8c98-4212-b1e5-80591bc6b117 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:20.577561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc1d62-8c98-4212-b1e5-80591bc6b117 None admin] PATH_INFO: `/v3/projects/5a58ca6c3969438e897eb2906823365f/users/91c44362773b4667825e3087cb73188e/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:20.579620 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc1d62-8c98-4212-b1e5-80591bc6b117 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a58ca6c3969438e897eb2906823365f, user_id=91c44362773b4667825e3087cb73188e, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:20.581185 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc1d62-8c98-4212-b1e5-80591bc6b117 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:20.596241 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 67/137] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:17:20 2022] PUT /identity/v3/projects/5a58ca6c3969438e897eb2906823365f/users/91c44362773b4667825e3087cb73188e/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:17:20.978589 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eafb9a-8720-4e00-9af3-0d16993e7b49 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:20.979136 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eafb9a-8720-4e00-9af3-0d16993e7b49 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:20.979209 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eafb9a-8720-4e00-9af3-0d16993e7b49 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:20.980845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 71/138] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:20.986294 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96550478-0ed2-4093-a7f8-879f15850cdf None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:20.986586 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96550478-0ed2-4093-a7f8-879f15850cdf None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:20.986865 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96550478-0ed2-4093-a7f8-879f15850cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:21.016347 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-96550478-0ed2-4093-a7f8-879f15850cdf None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:21.017538 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-96550478-0ed2-4093-a7f8-879f15850cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:21.035201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:21.095833 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 68/139] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:21.117336 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eafb9a-8720-4e00-9af3-0d16993e7b49 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:21.223545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eafb9a-8720-4e00-9af3-0d16993e7b49 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:21.258735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a633e9-7762-4e6e-ab61-8f53622d71ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:21.259297 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a633e9-7762-4e6e-ab61-8f53622d71ca None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:21.259577 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a633e9-7762-4e6e-ab61-8f53622d71ca None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:21.259849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a633e9-7762-4e6e-ab61-8f53622d71ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:21.270947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a633e9-7762-4e6e-ab61-8f53622d71ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:21.277213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-36a633e9-7762-4e6e-ab61-8f53622d71ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:17:21.277213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:21.277213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:21.277213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:21.277213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:21.277213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:21.277213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:21.277213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:21.277213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:21.277213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:21.277213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:21.277213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:21.277213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:21.277213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:21.277213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:21.277213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:21.278050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:21.278050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:21.278050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:21.278050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:21.278050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:21.278050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:21.278050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:21.278050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:21.278050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:21.278050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:21.278050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:21.278050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:21.278050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:21.278050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:21.278050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:21.278050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:21.278823 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:21.278823 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:21.278823 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:21.278823 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:21.278823 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:21.278823 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:21.278823 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:21.278823 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:21.278823 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:21.278823 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:21.278823 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:17:21.278823 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:21.281255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 72/140] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Aug 6 12:17:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:21.284948 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96550478-0ed2-4093-a7f8-879f15850cdf None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:21.413939 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96550478-0ed2-4093-a7f8-879f15850cdf None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:21.465009 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b8b0f-4d31-4730-af41-28d1ac054880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:21.465637 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b8b0f-4d31-4730-af41-28d1ac054880 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:21.465920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b8b0f-4d31-4730-af41-28d1ac054880 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:21.466194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b8b0f-4d31-4730-af41-28d1ac054880 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:21.466912 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b8b0f-4d31-4730-af41-28d1ac054880 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:21.467222 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b8b0f-4d31-4730-af41-28d1ac054880 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:21.468623 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b8b0f-4d31-4730-af41-28d1ac054880 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:21.480404 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 69/141] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:17:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:21.486014 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a633e9-7762-4e6e-ab61-8f53622d71ca None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:21.565978 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:21.625058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a633e9-7762-4e6e-ab61-8f53622d71ca None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:21.671864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a2fcb9-e9b6-4292-8319-2cbbbc03e079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:21.672539 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2fcb9-e9b6-4292-8319-2cbbbc03e079 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:21.672827 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2fcb9-e9b6-4292-8319-2cbbbc03e079 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:21.673108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2fcb9-e9b6-4292-8319-2cbbbc03e079 None admin] PATH_INFO: `/v3/users/91c44362773b4667825e3087cb73188e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:21.674316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a2fcb9-e9b6-4292-8319-2cbbbc03e079 None admin] RBAC: Authorizing `identity:get_user(user_id=91c44362773b4667825e3087cb73188e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:21.675783 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a2fcb9-e9b6-4292-8319-2cbbbc03e079 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:21.677661 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 73/142] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:21 2022] GET /identity/v3/users/91c44362773b4667825e3087cb73188e => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:21.683890 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b8b0f-4d31-4730-af41-28d1ac054880 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:21.707274 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:21.807530 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b8b0f-4d31-4730-af41-28d1ac054880 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:21.847400 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7c7f5-4f50-4fb6-8a02-53e07a925c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:21.847943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7c7f5-4f50-4fb6-8a02-53e07a925c16 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:21.848224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7c7f5-4f50-4fb6-8a02-53e07a925c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:21.848614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7c7f5-4f50-4fb6-8a02-53e07a925c16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:21.927997 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7c7f5-4f50-4fb6-8a02-53e07a925c16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:21.929554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7c7f5-4f50-4fb6-8a02-53e07a925c16 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:21.945150 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:22.049161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 70/143] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:22.055687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a2fcb9-e9b6-4292-8319-2cbbbc03e079 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:22.197425 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a2fcb9-e9b6-4292-8319-2cbbbc03e079 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:22.221435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:22.239119 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c16d6-a2a2-4ec3-8e33-4ac73985f9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:22.239768 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c16d6-a2a2-4ec3-8e33-4ac73985f9dd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:22.240057 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c16d6-a2a2-4ec3-8e33-4ac73985f9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:22.240615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c16d6-a2a2-4ec3-8e33-4ac73985f9dd None admin] PATH_INFO: `/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:22.241819 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c16d6-a2a2-4ec3-8e33-4ac73985f9dd None admin] RBAC: Authorizing `identity:get_project(project_id=5a84e3fab8334ecca0cd4aae83cb5317)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:22.243265 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c16d6-a2a2-4ec3-8e33-4ac73985f9dd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:22.245848 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 74/144] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:22 2022] GET /identity/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:22.251914 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7c7f5-4f50-4fb6-8a02-53e07a925c16 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:22.362836 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38f1b0-7445-46fb-b5e7-77fbd3a4483a None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:22.363190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38f1b0-7445-46fb-b5e7-77fbd3a4483a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:22.363423 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce38f1b0-7445-46fb-b5e7-77fbd3a4483a None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:22.364932 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 75/145] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:17:22.370901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47bd2e-f9a5-4692-9be3-c15afd7f7004 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:22.371187 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47bd2e-f9a5-4692-9be3-c15afd7f7004 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:22.371577 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47bd2e-f9a5-4692-9be3-c15afd7f7004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:22.383788 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7c7f5-4f50-4fb6-8a02-53e07a925c16 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:22.398531 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:22.409692 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ea47bd2e-f9a5-4692-9be3-c15afd7f7004 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:22.410904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ea47bd2e-f9a5-4692-9be3-c15afd7f7004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:22.460030 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97668578-bf1d-4994-bab9-2ed4be43631f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:22.460606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97668578-bf1d-4994-bab9-2ed4be43631f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:22.460889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97668578-bf1d-4994-bab9-2ed4be43631f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:22.461167 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97668578-bf1d-4994-bab9-2ed4be43631f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:22.461960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97668578-bf1d-4994-bab9-2ed4be43631f None admin] RBAC: Adding query filter params (role.id=47cde553d7ee465bbe685a05c00b94e2, scope.project.id=5a84e3fab8334ecca0cd4aae83cb5317, user.id=91c44362773b4667825e3087cb73188e) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:22.462271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97668578-bf1d-4994-bab9-2ed4be43631f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:22.464278 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97668578-bf1d-4994-bab9-2ed4be43631f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:22.477734 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 71/146] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:17:22 2022] GET /identity/v3/role_assignments?user.id=91c44362773b4667825e3087cb73188e&scope.project.id=5a84e3fab8334ecca0cd4aae83cb5317&role.id=47cde553d7ee465bbe685a05c00b94e2 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:22.481611 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573103f9-af51-44ff-a7fe-65bf54071c77 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:22.481914 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573103f9-af51-44ff-a7fe-65bf54071c77 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:22.482195 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573103f9-af51-44ff-a7fe-65bf54071c77 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:22.483571 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 72/147] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:22.489867 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29615d1-2c55-41f3-9d92-5c11902912f3 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:22.490160 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29615d1-2c55-41f3-9d92-5c11902912f3 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:22.490682 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29615d1-2c55-41f3-9d92-5c11902912f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:22.521913 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 76/148] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:22.522947 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-b29615d1-2c55-41f3-9d92-5c11902912f3 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:22.524190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b29615d1-2c55-41f3-9d92-5c11902912f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:22.554623 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea47bd2e-f9a5-4692-9be3-c15afd7f7004 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:22.616137 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 73/149] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:22.648633 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29615d1-2c55-41f3-9d92-5c11902912f3 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:22.709178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea47bd2e-f9a5-4692-9be3-c15afd7f7004 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:22.745639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:22.785081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4cfc35-17a2-41d9-a33d-322ebf56ab0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:22.785755 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cfc35-17a2-41d9-a33d-322ebf56ab0c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:22.786040 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cfc35-17a2-41d9-a33d-322ebf56ab0c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:22.786314 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cfc35-17a2-41d9-a33d-322ebf56ab0c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:22.793157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4cfc35-17a2-41d9-a33d-322ebf56ab0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:22.794642 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4cfc35-17a2-41d9-a33d-322ebf56ab0c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:22.796665 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 77/150] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:17:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:22.802612 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4cfc35-17a2-41d9-a33d-322ebf56ab0c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:22.845776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29615d1-2c55-41f3-9d92-5c11902912f3 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:22.862488 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:22.916213 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f43d68-bec4-4d50-acce-677fb9f94178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:22.917575 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f43d68-bec4-4d50-acce-677fb9f94178 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:22.918901 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f43d68-bec4-4d50-acce-677fb9f94178 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:22.920011 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f43d68-bec4-4d50-acce-677fb9f94178 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:22.948398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f43d68-bec4-4d50-acce-677fb9f94178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:22.954271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-a3f43d68-bec4-4d50-acce-677fb9f94178 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 12:17:22.954271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:22.954271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:22.954271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:22.954271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:22.954271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:22.954271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:22.954271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:22.954271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:22.954271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:22.954271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:22.954271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:22.954271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:22.954271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:22.954271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:22.954271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:22.955161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:22.955161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:22.955161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:22.955161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:22.955161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:22.955161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:22.955161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:22.955161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:22.955161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:22.955161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:22.955161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:22.955161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:22.955161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:22.955161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:22.955161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:22.955161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:22.955970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:22.955970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:22.955970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:22.955970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:22.955970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:22.955970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:22.955970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:22.955970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:22.955970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:22.955970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:22.955970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 12:17:22.955970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:17:22.960258 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 74/151] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:17:22.966561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f43d68-bec4-4d50-acce-677fb9f94178 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:23.014298 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4cfc35-17a2-41d9-a33d-322ebf56ab0c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:23.085573 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b64ae-3de5-4a72-86c6-288525241a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:23.086212 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b64ae-3de5-4a72-86c6-288525241a47 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:23.086475 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b64ae-3de5-4a72-86c6-288525241a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:23.086800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b64ae-3de5-4a72-86c6-288525241a47 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:23.087489 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b64ae-3de5-4a72-86c6-288525241a47 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:23.089492 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b64ae-3de5-4a72-86c6-288525241a47 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:23.109854 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f43d68-bec4-4d50-acce-677fb9f94178 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:23.119111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 78/152] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Aug 6 12:17:22 2022] POST /identity/v3/users => generated 283 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 12:17:23.121415 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367abff2-a1b9-4080-a337-ff52075f290b None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:23.121969 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367abff2-a1b9-4080-a337-ff52075f290b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:23.122448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367abff2-a1b9-4080-a337-ff52075f290b None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:23.123968 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 79/153] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:23.130227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d49150-2da0-4785-8989-d4219780589b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:23.130905 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d49150-2da0-4785-8989-d4219780589b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:23.131394 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d49150-2da0-4785-8989-d4219780589b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:23.151784 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c2d49150-2da0-4785-8989-d4219780589b None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:23.153335 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c2d49150-2da0-4785-8989-d4219780589b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:23.168344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077ec26-4e48-46ca-83cd-c6d8c399f3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:23.168924 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077ec26-4e48-46ca-83cd-c6d8c399f3a3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:23.169201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077ec26-4e48-46ca-83cd-c6d8c399f3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:23.169467 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077ec26-4e48-46ca-83cd-c6d8c399f3a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:23.170443 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077ec26-4e48-46ca-83cd-c6d8c399f3a3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:23.170641 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077ec26-4e48-46ca-83cd-c6d8c399f3a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:23.171967 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077ec26-4e48-46ca-83cd-c6d8c399f3a3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:23.185703 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 75/154] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:23.191420 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077ec26-4e48-46ca-83cd-c6d8c399f3a3 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:23.196095 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:23.233592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 80/155] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 12:17:23.247025 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:23.258722 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d49150-2da0-4785-8989-d4219780589b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:23.345287 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077ec26-4e48-46ca-83cd-c6d8c399f3a3 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:23.373435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d49150-2da0-4785-8989-d4219780589b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:23.393146 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e4776-a77f-4305-bd59-07dea3d3c98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:23.394423 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e4776-a77f-4305-bd59-07dea3d3c98f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:23.394774 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e4776-a77f-4305-bd59-07dea3d3c98f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:23.395120 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e4776-a77f-4305-bd59-07dea3d3c98f None admin] PATH_INFO: `/v3/users/f8d8e060c5cd456fa2c4a58592080111` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:23.396496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e4776-a77f-4305-bd59-07dea3d3c98f None admin] RBAC: Authorizing `identity:get_user(user_id=f8d8e060c5cd456fa2c4a58592080111)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:23.398899 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e4776-a77f-4305-bd59-07dea3d3c98f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:23.401914 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 76/156] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:23 2022] GET /identity/v3/users/f8d8e060c5cd456fa2c4a58592080111 => generated 289 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:23.403610 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7d20b-d169-4027-a3d9-4c9f07bfc6c1 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:23.404202 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7d20b-d169-4027-a3d9-4c9f07bfc6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:23.404504 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7d20b-d169-4027-a3d9-4c9f07bfc6c1 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:23.406996 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 77/157] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:23.407790 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7d20b-d169-4027-a3d9-4c9f07bfc6c1 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:23.421360 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99f01e-0c77-40d8-a8a0-55b1d06a10c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:23.422111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99f01e-0c77-40d8-a8a0-55b1d06a10c3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:23.422458 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99f01e-0c77-40d8-a8a0-55b1d06a10c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:23.422797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99f01e-0c77-40d8-a8a0-55b1d06a10c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:23.434747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de99f01e-0c77-40d8-a8a0-55b1d06a10c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:23.439756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-de99f01e-0c77-40d8-a8a0-55b1d06a10c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:23.439756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:23.439756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:23.439756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:23.439756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:23.439756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:23.439756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:23.439756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:23.439756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:23.439756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:23.439756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:23.439756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:23.439756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:23.439756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:23.439756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:23.439756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:23.440811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:23.440811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:23.440811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:23.440811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:23.440811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:23.440811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:23.440811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:23.440811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:23.440811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:23.440811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:23.440811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:23.440811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:23.440811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:23.440811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:23.440811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:23.440811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:23.441864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:23.441864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:23.441864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:23.441864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:23.441864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:23.441864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:23.441864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:23.441864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:23.441864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:23.441864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:23.441864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:23.441864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:23.441864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 81/158] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:23.443403 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e707ed-d90d-42f2-87af-fb5dbb16d9fc None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:23.443747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e707ed-d90d-42f2-87af-fb5dbb16d9fc None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:23.444088 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e707ed-d90d-42f2-87af-fb5dbb16d9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:23.462444 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d3e707ed-d90d-42f2-87af-fb5dbb16d9fc None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:23.463937 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d3e707ed-d90d-42f2-87af-fb5dbb16d9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:23.498831 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:23.537968 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:23.547240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7d20b-d169-4027-a3d9-4c9f07bfc6c1 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:23.556237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 82/159] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:23.557774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e707ed-d90d-42f2-87af-fb5dbb16d9fc None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:23.590065 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08781a27-838a-4213-a3aa-0d734d17954d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:23.591239 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08781a27-838a-4213-a3aa-0d734d17954d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:23.591521 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08781a27-838a-4213-a3aa-0d734d17954d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:23.591795 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08781a27-838a-4213-a3aa-0d734d17954d None admin] PATH_INFO: `/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:23.593132 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08781a27-838a-4213-a3aa-0d734d17954d None admin] RBAC: Authorizing `identity:get_project(project_id=5a84e3fab8334ecca0cd4aae83cb5317)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:23.594596 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08781a27-838a-4213-a3aa-0d734d17954d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:23.596836 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 78/160] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:23 2022] GET /identity/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:23.598007 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08781a27-838a-4213-a3aa-0d734d17954d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:23.696884 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e707ed-d90d-42f2-87af-fb5dbb16d9fc None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:23.738739 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e05e3-bb0c-489e-b467-38f48a374118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:23.739362 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e05e3-bb0c-489e-b467-38f48a374118 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:23.739637 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e05e3-bb0c-489e-b467-38f48a374118 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:23.739905 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e05e3-bb0c-489e-b467-38f48a374118 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:23.740802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08781a27-838a-4213-a3aa-0d734d17954d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:23.740908 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963e05e3-bb0c-489e-b467-38f48a374118 None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:23.741197 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963e05e3-bb0c-489e-b467-38f48a374118 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:23.745596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963e05e3-bb0c-489e-b467-38f48a374118 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:23.756189 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 83/161] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:23.757636 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e05e3-bb0c-489e-b467-38f48a374118 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:23.786330 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a866f-bae2-4a64-a218-cb912d22f3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:23.787096 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a866f-bae2-4a64-a218-cb912d22f3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:23.787695 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:23.788414 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a866f-bae2-4a64-a218-cb912d22f3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:23.788756 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a866f-bae2-4a64-a218-cb912d22f3d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:23.802163 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a866f-bae2-4a64-a218-cb912d22f3d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:23.807758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-688a866f-bae2-4a64-a218-cb912d22f3d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:17:23.807758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:23.807758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:23.807758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:23.807758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:23.807758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:23.807758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:23.807758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:23.807758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:23.807758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:23.807758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:23.807758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:23.807758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:23.807758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:23.807758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:23.807758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:23.808751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:23.808751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:23.808751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:23.808751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:23.808751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:23.808751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:23.808751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:23.808751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:23.808751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:23.808751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:23.808751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:23.808751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:23.808751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:23.808751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:23.808751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:23.808751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:23.809695 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:23.809695 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:23.809695 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:23.809695 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:23.809695 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:23.809695 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:23.809695 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:23.809695 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:23.809695 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:23.809695 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:23.809695 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:17:23.809695 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:17:23.810576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 79/162] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:17:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:23.811757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286c61e-6aff-49d0-93f8-c395640deed0 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:23.812094 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286c61e-6aff-49d0-93f8-c395640deed0 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:23.818517 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286c61e-6aff-49d0-93f8-c395640deed0 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:23.820186 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 80/163] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:23.820835 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286c61e-6aff-49d0-93f8-c395640deed0 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:23.862513 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:23.894651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e05e3-bb0c-489e-b467-38f48a374118 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:23.944256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb52de9-6586-4ab7-afd0-057aff01ee30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:23.945067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb52de9-6586-4ab7-afd0-057aff01ee30 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:23.945415 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb52de9-6586-4ab7-afd0-057aff01ee30 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:23.946012 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb52de9-6586-4ab7-afd0-057aff01ee30 None admin] PATH_INFO: `/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317/users/f8d8e060c5cd456fa2c4a58592080111/roles/7f46f07e73294f919cb56f7a97b324d7` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:23.948884 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb52de9-6586-4ab7-afd0-057aff01ee30 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a84e3fab8334ecca0cd4aae83cb5317, user_id=f8d8e060c5cd456fa2c4a58592080111, role_id=7f46f07e73294f919cb56f7a97b324d7)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:23.951095 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb52de9-6586-4ab7-afd0-057aff01ee30 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:23.957397 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286c61e-6aff-49d0-93f8-c395640deed0 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:23.983445 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 84/164] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:17:23 2022] PUT /identity/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317/users/f8d8e060c5cd456fa2c4a58592080111/roles/7f46f07e73294f919cb56f7a97b324d7 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:17:23.984157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb52de9-6586-4ab7-afd0-057aff01ee30 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:24.018873 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c0a1d-5c12-4cae-b479-00c1ed4dcb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:24.019848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c0a1d-5c12-4cae-b479-00c1ed4dcb22 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:24.020261 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c0a1d-5c12-4cae-b479-00c1ed4dcb22 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:24.020904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c0a1d-5c12-4cae-b479-00c1ed4dcb22 None admin] PATH_INFO: `/v3/users/91c44362773b4667825e3087cb73188e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:24.022764 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c0a1d-5c12-4cae-b479-00c1ed4dcb22 None admin] RBAC: Authorizing `identity:get_user(user_id=91c44362773b4667825e3087cb73188e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:24.025045 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c0a1d-5c12-4cae-b479-00c1ed4dcb22 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:24.027703 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 81/165] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:23 2022] GET /identity/v3/users/91c44362773b4667825e3087cb73188e => generated 275 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:24.029418 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6d62b-96f9-4c4f-a69c-a92115e440df None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:24.030056 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6d62b-96f9-4c4f-a69c-a92115e440df None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:24.030475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6d62b-96f9-4c4f-a69c-a92115e440df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:24.061317 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-12e6d62b-96f9-4c4f-a69c-a92115e440df None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:24.062930 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-12e6d62b-96f9-4c4f-a69c-a92115e440df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:24.147681 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 82/166] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:24.149741 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6d62b-96f9-4c4f-a69c-a92115e440df None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:24.185064 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:24.199009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:24.252346 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb52de9-6586-4ab7-afd0-057aff01ee30 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:24.313126 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6d62b-96f9-4c4f-a69c-a92115e440df None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:24.353222 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d4562-4ed6-44f3-9c28-a66998199e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:24.354405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d4562-4ed6-44f3-9c28-a66998199e3b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:24.354685 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d4562-4ed6-44f3-9c28-a66998199e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:24.354852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0e4f7-3dc1-4e33-870c-c8a38a68d99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:24.354974 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d4562-4ed6-44f3-9c28-a66998199e3b None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:24.355464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0e4f7-3dc1-4e33-870c-c8a38a68d99e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:24.355672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d4562-4ed6-44f3-9c28-a66998199e3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:24.355773 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0e4f7-3dc1-4e33-870c-c8a38a68d99e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:24.355975 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d4562-4ed6-44f3-9c28-a66998199e3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:24.356079 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0e4f7-3dc1-4e33-870c-c8a38a68d99e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:24.357404 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d4562-4ed6-44f3-9c28-a66998199e3b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:24.372586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 85/167] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:24.375818 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d4562-4ed6-44f3-9c28-a66998199e3b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:24.493986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0e4f7-3dc1-4e33-870c-c8a38a68d99e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:24.496196 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0e4f7-3dc1-4e33-870c-c8a38a68d99e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:24.548681 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:24.561047 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d4562-4ed6-44f3-9c28-a66998199e3b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:24.637726 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85918626-d563-435e-9d69-b982de04f484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:24.638304 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85918626-d563-435e-9d69-b982de04f484 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:24.638586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85918626-d563-435e-9d69-b982de04f484 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:24.638863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85918626-d563-435e-9d69-b982de04f484 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:24.662101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85918626-d563-435e-9d69-b982de04f484 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:24.670316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-85918626-d563-435e-9d69-b982de04f484 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:24.670316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:24.670316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:24.670316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:24.670316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:24.670316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:24.670316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:24.670316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:24.670316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:24.670316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:24.670316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:24.670316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:24.670316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:24.670316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:24.670316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:24.670316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:24.671264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:24.671264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:24.671264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:24.671264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:24.671264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:24.671264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:24.671264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:24.671264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:24.671264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:24.671264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:24.671264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:24.671264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:24.671264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:24.671264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:24.671264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:24.671264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:24.672021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:24.672021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:24.672021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:24.672021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:24.672021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:24.672021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:24.672021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:24.672021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:24.672021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:24.672021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:24.672021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:24.672021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:24.673715 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 86/168] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:24.674506 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85918626-d563-435e-9d69-b982de04f484 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:24.695612 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:24.708189 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 83/169] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:24.709087 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0e4f7-3dc1-4e33-870c-c8a38a68d99e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:24.889260 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85918626-d563-435e-9d69-b982de04f484 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:24.953894 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0e4f7-3dc1-4e33-870c-c8a38a68d99e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:24.982038 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d7d67-156f-4947-990f-bb3a40aef3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:24.984131 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d7d67-156f-4947-990f-bb3a40aef3bf None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:24.984634 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d7d67-156f-4947-990f-bb3a40aef3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:24.985100 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d7d67-156f-4947-990f-bb3a40aef3bf None admin] PATH_INFO: `/v3/users/c9de148de73e459eb6456afdaf9fe213` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:24.986646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d7d67-156f-4947-990f-bb3a40aef3bf None admin] RBAC: Authorizing `identity:get_user(user_id=c9de148de73e459eb6456afdaf9fe213)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:24.989020 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d7d67-156f-4947-990f-bb3a40aef3bf None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:25.004221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 87/170] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:24 2022] GET /identity/v3/users/c9de148de73e459eb6456afdaf9fe213 => generated 247 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:25.005403 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d7d67-156f-4947-990f-bb3a40aef3bf None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:25.024969 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:25.046498 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a6a3d-b464-4af1-8d89-1d9318248a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:25.048413 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a6a3d-b464-4af1-8d89-1d9318248a1e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:25.051806 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a6a3d-b464-4af1-8d89-1d9318248a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:25.052404 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a6a3d-b464-4af1-8d89-1d9318248a1e None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:25.053333 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a6a3d-b464-4af1-8d89-1d9318248a1e None admin] RBAC: Adding query filter params (name=member) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:25.053826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a6a3d-b464-4af1-8d89-1d9318248a1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:25.055417 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a6a3d-b464-4af1-8d89-1d9318248a1e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:25.075651 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 84/171] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:25.078052 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a6a3d-b464-4af1-8d89-1d9318248a1e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:25.164819 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:25.217204 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d7d67-156f-4947-990f-bb3a40aef3bf None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:25.283676 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a6a3d-b464-4af1-8d89-1d9318248a1e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:25.308065 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446ccb3-60bf-4299-bbff-f8457c12f008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:25.309061 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446ccb3-60bf-4299-bbff-f8457c12f008 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:25.309555 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446ccb3-60bf-4299-bbff-f8457c12f008 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:25.310027 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446ccb3-60bf-4299-bbff-f8457c12f008 None admin] PATH_INFO: `/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:25.311875 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446ccb3-60bf-4299-bbff-f8457c12f008 None admin] RBAC: Authorizing `identity:get_project(project_id=5a84e3fab8334ecca0cd4aae83cb5317)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:25.313534 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446ccb3-60bf-4299-bbff-f8457c12f008 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:25.315601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 88/172] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:25 2022] GET /identity/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317 => generated 298 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:25.318675 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446ccb3-60bf-4299-bbff-f8457c12f008 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:25.362930 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f6407-fbae-4561-ad23-779ed1c84ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:25.364110 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f6407-fbae-4561-ad23-779ed1c84ffc None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:25.364627 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f6407-fbae-4561-ad23-779ed1c84ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:25.365291 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f6407-fbae-4561-ad23-779ed1c84ffc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:25.472153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446ccb3-60bf-4299-bbff-f8457c12f008 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:25.486945 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:25.517051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f6407-fbae-4561-ad23-779ed1c84ffc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:25.518547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f6407-fbae-4561-ad23-779ed1c84ffc None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:25.519878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a75ce-f102-4899-8e26-dba842f4ef6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:25.520551 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a75ce-f102-4899-8e26-dba842f4ef6b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:25.520829 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a75ce-f102-4899-8e26-dba842f4ef6b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:25.521099 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a75ce-f102-4899-8e26-dba842f4ef6b None admin] PATH_INFO: `/v3/users/91c44362773b4667825e3087cb73188e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:25.523414 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a75ce-f102-4899-8e26-dba842f4ef6b None admin] RBAC: Authorizing `identity:get_user(user_id=91c44362773b4667825e3087cb73188e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:25.524890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a75ce-f102-4899-8e26-dba842f4ef6b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:25.526882 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 89/173] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:25 2022] GET /identity/v3/users/91c44362773b4667825e3087cb73188e => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:25.527671 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a75ce-f102-4899-8e26-dba842f4ef6b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:25.557705 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:25.707503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 85/174] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:25.712482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f6407-fbae-4561-ad23-779ed1c84ffc None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:25.740369 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a75ce-f102-4899-8e26-dba842f4ef6b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:25.784468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de24907-31e6-4369-b1df-3599806228b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:25.785096 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de24907-31e6-4369-b1df-3599806228b5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:25.785380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de24907-31e6-4369-b1df-3599806228b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:25.785659 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de24907-31e6-4369-b1df-3599806228b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:25.786456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de24907-31e6-4369-b1df-3599806228b5 None admin] RBAC: Adding query filter params (role.id=0fd7b75299b64c54971d9bfeaec30f4e, scope.project.id=5a84e3fab8334ecca0cd4aae83cb5317, user.id=91c44362773b4667825e3087cb73188e) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:25.786774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de24907-31e6-4369-b1df-3599806228b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:25.788275 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de24907-31e6-4369-b1df-3599806228b5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:25.795326 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 90/175] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:17:25 2022] GET /identity/v3/role_assignments?user.id=91c44362773b4667825e3087cb73188e&scope.project.id=5a84e3fab8334ecca0cd4aae83cb5317&role.id=0fd7b75299b64c54971d9bfeaec30f4e => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:25.796769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de24907-31e6-4369-b1df-3599806228b5 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:25.861878 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:25.888669 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:25.895896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f6407-fbae-4561-ad23-779ed1c84ffc None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:25.941756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de24907-31e6-4369-b1df-3599806228b5 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:25.965896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a31dd-897a-490d-8b99-b8eb12cdcd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:25.966525 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a31dd-897a-490d-8b99-b8eb12cdcd66 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:25.966808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a31dd-897a-490d-8b99-b8eb12cdcd66 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:25.967080 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a31dd-897a-490d-8b99-b8eb12cdcd66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:26.015491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b22b42-dc0f-418f-8e37-f073e2df2914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:26.016152 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b22b42-dc0f-418f-8e37-f073e2df2914 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:26.016460 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b22b42-dc0f-418f-8e37-f073e2df2914 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:26.016735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b22b42-dc0f-418f-8e37-f073e2df2914 None admin] PATH_INFO: `/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:26.019400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b22b42-dc0f-418f-8e37-f073e2df2914 None admin] RBAC: Authorizing `identity:get_project(project_id=5a84e3fab8334ecca0cd4aae83cb5317)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:26.020904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b22b42-dc0f-418f-8e37-f073e2df2914 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:26.023761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 91/176] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:25 2022] GET /identity/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:26.028711 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b22b42-dc0f-418f-8e37-f073e2df2914 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:26.114733 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a31dd-897a-490d-8b99-b8eb12cdcd66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:26.116507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a31dd-897a-490d-8b99-b8eb12cdcd66 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:26.196271 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b22b42-dc0f-418f-8e37-f073e2df2914 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:26.262023 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347c6df-066e-4c06-a765-32ec178e6917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:26.262632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347c6df-066e-4c06-a765-32ec178e6917 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:26.262867 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347c6df-066e-4c06-a765-32ec178e6917 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:26.263116 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347c6df-066e-4c06-a765-32ec178e6917 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:26.263923 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347c6df-066e-4c06-a765-32ec178e6917 None admin] RBAC: Adding query filter params (role.id=144c8425688b409384e1e8ed8279d88e, scope.project.id=5a84e3fab8334ecca0cd4aae83cb5317, user.id=c9de148de73e459eb6456afdaf9fe213) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:26.264233 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347c6df-066e-4c06-a765-32ec178e6917 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:26.265736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347c6df-066e-4c06-a765-32ec178e6917 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:26.277795 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:26.282986 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 92/177] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:17:26 2022] GET /identity/v3/role_assignments?user.id=c9de148de73e459eb6456afdaf9fe213&scope.project.id=5a84e3fab8334ecca0cd4aae83cb5317&role.id=144c8425688b409384e1e8ed8279d88e => generated 608 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:26.300084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 86/178] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:26.305884 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347c6df-066e-4c06-a765-32ec178e6917 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:26.318406 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:26.411029 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9a498-4f79-4cda-a066-6c405b1343f0 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:26.411347 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9a498-4f79-4cda-a066-6c405b1343f0 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:26.411595 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9a498-4f79-4cda-a066-6c405b1343f0 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:26.413552 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 87/179] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:26.422398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537cef9-5cc1-4601-9cbb-f0bcc2d9b07d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:26.422695 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537cef9-5cc1-4601-9cbb-f0bcc2d9b07d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:26.422945 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537cef9-5cc1-4601-9cbb-f0bcc2d9b07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:26.457830 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-d537cef9-5cc1-4601-9cbb-f0bcc2d9b07d None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:26.459061 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d537cef9-5cc1-4601-9cbb-f0bcc2d9b07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:26.476883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347c6df-066e-4c06-a765-32ec178e6917 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:26.559473 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfffc672-ad5e-4a9b-b58e-932766ec4ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:26.560200 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfffc672-ad5e-4a9b-b58e-932766ec4ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:26.560509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfffc672-ad5e-4a9b-b58e-932766ec4ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:26.560786 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfffc672-ad5e-4a9b-b58e-932766ec4ea2 None admin] PATH_INFO: `/v3/projects/5a58ca6c3969438e897eb2906823365f` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:26.562052 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfffc672-ad5e-4a9b-b58e-932766ec4ea2 None admin] RBAC: Authorizing `identity:get_project(project_id=5a58ca6c3969438e897eb2906823365f)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:26.563715 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfffc672-ad5e-4a9b-b58e-932766ec4ea2 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:26.567817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 93/180] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:26 2022] GET /identity/v3/projects/5a58ca6c3969438e897eb2906823365f => generated 312 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:26.573545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18586fa-cf15-4a0b-9128-54204371c953 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:26.574586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18586fa-cf15-4a0b-9128-54204371c953 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:26.574865 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18586fa-cf15-4a0b-9128-54204371c953 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:26.576226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 94/181] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:26.576984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18586fa-cf15-4a0b-9128-54204371c953 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:26.612933 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 88/182] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 12:17:26.614429 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed7fb6-ea7c-47fd-bf5a-0921c3783d8b None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:26.614812 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed7fb6-ea7c-47fd-bf5a-0921c3783d8b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:26.615089 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed7fb6-ea7c-47fd-bf5a-0921c3783d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:26.644824 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-37ed7fb6-ea7c-47fd-bf5a-0921c3783d8b None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:26.646065 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-37ed7fb6-ea7c-47fd-bf5a-0921c3783d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:26.675607 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:26.707971 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18586fa-cf15-4a0b-9128-54204371c953 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:26.745259 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 89/183] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:26.745481 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2a67a-3d38-4fec-9a25-21984e93a101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:26.745790 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2a67a-3d38-4fec-9a25-21984e93a101 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:26.745887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed7fb6-ea7c-47fd-bf5a-0921c3783d8b None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:26.746067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2a67a-3d38-4fec-9a25-21984e93a101 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:26.746338 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2a67a-3d38-4fec-9a25-21984e93a101 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:26.747176 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2a67a-3d38-4fec-9a25-21984e93a101 None admin] RBAC: Adding query filter params (role.id=0fd7b75299b64c54971d9bfeaec30f4e, scope.project.id=5a58ca6c3969438e897eb2906823365f, user.id=91c44362773b4667825e3087cb73188e) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:26.747449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2a67a-3d38-4fec-9a25-21984e93a101 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:26.748915 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2a67a-3d38-4fec-9a25-21984e93a101 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:26.756358 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 95/184] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:17:26 2022] GET /identity/v3/role_assignments?user.id=91c44362773b4667825e3087cb73188e&scope.project.id=5a58ca6c3969438e897eb2906823365f&role.id=0fd7b75299b64c54971d9bfeaec30f4e => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:26.757581 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e402220-3ea6-43bc-997e-70dcc03f9879 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:26.757868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e402220-3ea6-43bc-997e-70dcc03f9879 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:26.758147 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e402220-3ea6-43bc-997e-70dcc03f9879 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:26.759366 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 96/185] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:26.765160 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab027ef-a141-454e-ac29-f41315ab5240 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:26.765482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab027ef-a141-454e-ac29-f41315ab5240 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:26.765767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab027ef-a141-454e-ac29-f41315ab5240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:26.785905 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6ab027ef-a141-454e-ac29-f41315ab5240 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:26.787134 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6ab027ef-a141-454e-ac29-f41315ab5240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:26.803550 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:26.869531 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 97/186] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 12:17:26.870344 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab027ef-a141-454e-ac29-f41315ab5240 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:26.875102 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed7fb6-ea7c-47fd-bf5a-0921c3783d8b None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:26.930035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf431f7-90d7-485a-880c-d215557597ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:26.930753 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf431f7-90d7-485a-880c-d215557597ce None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:26.931043 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf431f7-90d7-485a-880c-d215557597ce None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:26.931320 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf431f7-90d7-485a-880c-d215557597ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:26.946672 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf431f7-90d7-485a-880c-d215557597ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:26.954965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-4cf431f7-90d7-485a-880c-d215557597ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:17:26.954965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:26.954965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:26.954965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:26.954965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:26.954965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:26.954965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:26.954965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:26.954965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:26.954965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:26.954965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:26.954965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:26.954965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:26.954965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:26.954965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:26.954965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:26.955874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:26.955874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:26.955874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:26.955874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:26.955874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:26.955874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:26.955874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:26.955874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:26.955874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:26.955874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:26.955874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:26.955874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:26.955874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:26.955874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:26.955874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:26.955874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:26.961770 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:26.961770 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:26.961770 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:26.961770 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:26.961770 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:26.961770 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:26.961770 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:26.961770 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:26.961770 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:26.961770 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:26.961770 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:17:26.961770 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:17:26.961770 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 90/187] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:17:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:26.961770 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf431f7-90d7-485a-880c-d215557597ce None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:26.981973 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:27.028343 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab027ef-a141-454e-ac29-f41315ab5240 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:27.077046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8725bef-a84d-40a0-96aa-324effdf479d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:27.078256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8725bef-a84d-40a0-96aa-324effdf479d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:27.078860 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8725bef-a84d-40a0-96aa-324effdf479d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:27.079146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8725bef-a84d-40a0-96aa-324effdf479d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:27.081499 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf431f7-90d7-485a-880c-d215557597ce None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:27.090171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:27.096125 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8725bef-a84d-40a0-96aa-324effdf479d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:27.101227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-c8725bef-a84d-40a0-96aa-324effdf479d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:17:27.101227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:27.101227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:27.101227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:27.101227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:27.101227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:27.101227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:27.101227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:27.101227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:27.101227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:27.101227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:27.101227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:27.101227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:27.101227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:27.101227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:27.101227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:27.102086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:27.102086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:27.102086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:27.102086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:27.102086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:27.102086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:27.102086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:27.102086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:27.102086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:27.102086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:27.102086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:27.102086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:27.102086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:27.102086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:27.102086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:27.102086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:27.102877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:27.102877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:27.102877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:27.102877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:27.102877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:27.102877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:27.102877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:27.102877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:27.102877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:27.102877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:27.102877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:17:27.102877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:27.102877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 98/188] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Aug 6 12:17:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:27.103545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8725bef-a84d-40a0-96aa-324effdf479d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:27.126336 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1967e2a-3f55-4f7d-aef3-cbc56057963d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:27.126893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1967e2a-3f55-4f7d-aef3-cbc56057963d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:27.127166 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1967e2a-3f55-4f7d-aef3-cbc56057963d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:27.127433 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1967e2a-3f55-4f7d-aef3-cbc56057963d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:27.128627 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1967e2a-3f55-4f7d-aef3-cbc56057963d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:27.130075 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1967e2a-3f55-4f7d-aef3-cbc56057963d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:27.131788 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 91/189] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:17:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:27.132491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1967e2a-3f55-4f7d-aef3-cbc56057963d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:27.214199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8725bef-a84d-40a0-96aa-324effdf479d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:27.230891 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1967e2a-3f55-4f7d-aef3-cbc56057963d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:27.240221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:27.249526 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f95f3-4efc-49e9-8112-ef3b28268502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:27.250145 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f95f3-4efc-49e9-8112-ef3b28268502 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:27.250428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f95f3-4efc-49e9-8112-ef3b28268502 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:27.250704 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f95f3-4efc-49e9-8112-ef3b28268502 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:27.251419 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f95f3-4efc-49e9-8112-ef3b28268502 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:27.251730 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f95f3-4efc-49e9-8112-ef3b28268502 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:27.253144 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f95f3-4efc-49e9-8112-ef3b28268502 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:27.261388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 99/190] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:27.261989 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f95f3-4efc-49e9-8112-ef3b28268502 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:27.269087 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a47a4-8091-4bb7-b55f-815bf9edeb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:27.269715 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a47a4-8091-4bb7-b55f-815bf9edeb37 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:27.269970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a47a4-8091-4bb7-b55f-815bf9edeb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:27.270237 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a47a4-8091-4bb7-b55f-815bf9edeb37 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:27.270947 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a47a4-8091-4bb7-b55f-815bf9edeb37 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:27.271244 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a47a4-8091-4bb7-b55f-815bf9edeb37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:27.272653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a47a4-8091-4bb7-b55f-815bf9edeb37 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:27.280980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 92/191] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:17:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:27.282306 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e1c14-9aaf-402a-b63a-f61f30384756 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:27.282595 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e1c14-9aaf-402a-b63a-f61f30384756 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:27.282877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e1c14-9aaf-402a-b63a-f61f30384756 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:27.284346 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 93/192] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:27.284813 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e1c14-9aaf-402a-b63a-f61f30384756 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:27.319502 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:27.367130 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f95f3-4efc-49e9-8112-ef3b28268502 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:27.385602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e1c14-9aaf-402a-b63a-f61f30384756 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:27.401533 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc630c-b093-41f1-a8b9-2919d26844f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:27.402158 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc630c-b093-41f1-a8b9-2919d26844f4 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:27.402439 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc630c-b093-41f1-a8b9-2919d26844f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:27.402713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc630c-b093-41f1-a8b9-2919d26844f4 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:27.403390 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc630c-b093-41f1-a8b9-2919d26844f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:27.404880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc630c-b093-41f1-a8b9-2919d26844f4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:27.419607 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e0e07-46b4-4793-a725-46f0f4de84cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:27.420244 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e0e07-46b4-4793-a725-46f0f4de84cf None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:27.420550 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e0e07-46b4-4793-a725-46f0f4de84cf None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:27.420825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e0e07-46b4-4793-a725-46f0f4de84cf None admin] PATH_INFO: `/v3/users/3d958849e02b4ae8bcb375e7da0949e2` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:27.425086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 100/193] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Aug 6 12:17:27 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:17:27.425086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc630c-b093-41f1-a8b9-2919d26844f4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:27.435496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e0e07-46b4-4793-a725-46f0f4de84cf None admin] RBAC: Authorizing `identity:get_user(user_id=3d958849e02b4ae8bcb375e7da0949e2)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:27.437129 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e0e07-46b4-4793-a725-46f0f4de84cf None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:27.440351 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 94/194] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:27 2022] GET /identity/v3/users/3d958849e02b4ae8bcb375e7da0949e2 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:27.440351 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420aa597-9a69-4e72-a4d2-fad8dae649ce None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:27.440967 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420aa597-9a69-4e72-a4d2-fad8dae649ce None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:27.440967 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420aa597-9a69-4e72-a4d2-fad8dae649ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:27.460365 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-420aa597-9a69-4e72-a4d2-fad8dae649ce None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:27.461588 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-420aa597-9a69-4e72-a4d2-fad8dae649ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:27.492962 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:27.537848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 95/195] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:27.549357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420aa597-9a69-4e72-a4d2-fad8dae649ce None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:27.561067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc630c-b093-41f1-a8b9-2919d26844f4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:27.578090 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:27.601441 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bda19-17ca-4030-8ea7-efd07de50782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:27.602432 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bda19-17ca-4030-8ea7-efd07de50782 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:27.602998 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bda19-17ca-4030-8ea7-efd07de50782 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:27.603362 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bda19-17ca-4030-8ea7-efd07de50782 None admin] PATH_INFO: `/v3/users/3d958849e02b4ae8bcb375e7da0949e2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:27.604819 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5bda19-17ca-4030-8ea7-efd07de50782 None admin] RBAC: Authorizing `identity:get_user(user_id=3d958849e02b4ae8bcb375e7da0949e2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:27.606268 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5bda19-17ca-4030-8ea7-efd07de50782 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:27.608195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 101/196] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:27 2022] GET /identity/v3/users/3d958849e02b4ae8bcb375e7da0949e2 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:27.610866 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bda19-17ca-4030-8ea7-efd07de50782 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:27.668879 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420aa597-9a69-4e72-a4d2-fad8dae649ce None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:27.716761 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7a8e6-37f9-413e-9b24-c990deaaf246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:27.717793 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7a8e6-37f9-413e-9b24-c990deaaf246 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:27.718082 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7a8e6-37f9-413e-9b24-c990deaaf246 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:27.718360 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7a8e6-37f9-413e-9b24-c990deaaf246 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:27.740143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bda19-17ca-4030-8ea7-efd07de50782 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:27.761348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:27.792491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb7eb57-9ec5-4a83-b984-5ca586ab939c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:27.793391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7eb57-9ec5-4a83-b984-5ca586ab939c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:27.793673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7eb57-9ec5-4a83-b984-5ca586ab939c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:27.793947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7eb57-9ec5-4a83-b984-5ca586ab939c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:27.805872 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7a8e6-37f9-413e-9b24-c990deaaf246 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:27.807563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb7eb57-9ec5-4a83-b984-5ca586ab939c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:27.808256 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7a8e6-37f9-413e-9b24-c990deaaf246 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:27.814672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-5fb7eb57-9ec5-4a83-b984-5ca586ab939c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 12:17:27.814672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:27.814672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:27.814672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:27.814672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:27.814672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:27.814672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:27.814672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:27.814672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:27.814672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:27.814672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:27.814672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:27.814672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:27.814672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:27.814672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:27.814672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:27.815545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:27.815545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:27.815545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:27.815545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:27.815545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:27.815545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:27.815545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:27.815545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:27.815545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:27.815545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:27.815545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:27.815545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:27.815545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:27.815545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:27.815545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:27.815545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:27.816856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:27.816856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:27.816856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:27.816856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:27.816856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:27.816856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:27.816856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:27.816856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:27.816856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:27.816856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:27.816856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 12:17:27.816856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:27.816856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 102/197] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:27.816856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb7eb57-9ec5-4a83-b984-5ca586ab939c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:27.842341 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:27.942588 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb7eb57-9ec5-4a83-b984-5ca586ab939c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:27.943158 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 96/198] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:27.944015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7a8e6-37f9-413e-9b24-c990deaaf246 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:27.980640 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e4159-c150-4437-95b3-7984a30dfc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:27.981620 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e4159-c150-4437-95b3-7984a30dfc44 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:27.982150 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e4159-c150-4437-95b3-7984a30dfc44 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:27.982611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e4159-c150-4437-95b3-7984a30dfc44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:28.031884 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:28.061527 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e4159-c150-4437-95b3-7984a30dfc44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:28.063303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e4159-c150-4437-95b3-7984a30dfc44 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:28.068841 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7a8e6-37f9-413e-9b24-c990deaaf246 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:28.096102 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:28.109563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16553f6-4c94-43bb-8b65-5ad31dba006b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:28.110534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16553f6-4c94-43bb-8b65-5ad31dba006b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:28.111005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16553f6-4c94-43bb-8b65-5ad31dba006b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:28.111490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16553f6-4c94-43bb-8b65-5ad31dba006b None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:28.112422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16553f6-4c94-43bb-8b65-5ad31dba006b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:28.112915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16553f6-4c94-43bb-8b65-5ad31dba006b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:28.114621 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16553f6-4c94-43bb-8b65-5ad31dba006b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:28.124687 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 97/199] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:28.126041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16553f6-4c94-43bb-8b65-5ad31dba006b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:28.198717 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 103/200] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:28.199901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e4159-c150-4437-95b3-7984a30dfc44 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:28.253263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16553f6-4c94-43bb-8b65-5ad31dba006b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:28.285833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:28.290458 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49329d8e-ea40-4262-b0d5-99032edb6671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:28.291114 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49329d8e-ea40-4262-b0d5-99032edb6671 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:28.291425 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49329d8e-ea40-4262-b0d5-99032edb6671 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:28.291702 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49329d8e-ea40-4262-b0d5-99032edb6671 None admin] PATH_INFO: `/v3/projects/34ca99f1da904d608e324f65f6571250` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:28.293276 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49329d8e-ea40-4262-b0d5-99032edb6671 None admin] RBAC: Authorizing `identity:get_project(project_id=34ca99f1da904d608e324f65f6571250)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:28.294987 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49329d8e-ea40-4262-b0d5-99032edb6671 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:28.297436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 98/201] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:28 2022] GET /identity/v3/projects/34ca99f1da904d608e324f65f6571250 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:28.298026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49329d8e-ea40-4262-b0d5-99032edb6671 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:28.325826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e4159-c150-4437-95b3-7984a30dfc44 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:28.363823 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:28.378079 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966aa25-f95e-4893-b631-67bcc5da8d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:28.378715 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966aa25-f95e-4893-b631-67bcc5da8d4b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:28.378986 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966aa25-f95e-4893-b631-67bcc5da8d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:28.379252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966aa25-f95e-4893-b631-67bcc5da8d4b None admin] PATH_INFO: `/v3/users/f8d8e060c5cd456fa2c4a58592080111` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:28.381073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966aa25-f95e-4893-b631-67bcc5da8d4b None admin] RBAC: Authorizing `identity:get_user(user_id=f8d8e060c5cd456fa2c4a58592080111)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:28.383612 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6966aa25-f95e-4893-b631-67bcc5da8d4b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:28.386371 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 104/202] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:28 2022] GET /identity/v3/users/f8d8e060c5cd456fa2c4a58592080111 => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:28.388414 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966aa25-f95e-4893-b631-67bcc5da8d4b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:28.419266 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49329d8e-ea40-4262-b0d5-99032edb6671 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:28.454944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa5e35-bdf5-439a-bbf9-f9e8738ddef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:28.455492 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa5e35-bdf5-439a-bbf9-f9e8738ddef2 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:28.455768 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa5e35-bdf5-439a-bbf9-f9e8738ddef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:28.456044 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa5e35-bdf5-439a-bbf9-f9e8738ddef2 None admin] PATH_INFO: `/v3/projects/34ca99f1da904d608e324f65f6571250` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:28.457426 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa5e35-bdf5-439a-bbf9-f9e8738ddef2 None admin] RBAC: Authorizing `identity:get_project(project_id=34ca99f1da904d608e324f65f6571250)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:28.458639 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa5e35-bdf5-439a-bbf9-f9e8738ddef2 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:28.465583 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 99/203] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:28 2022] GET /identity/v3/projects/34ca99f1da904d608e324f65f6571250 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:28.466032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa5e35-bdf5-439a-bbf9-f9e8738ddef2 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:28.501128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6966aa25-f95e-4893-b631-67bcc5da8d4b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:28.536856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95bcbd-cbc0-4aa9-8120-6807ba66f849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:28.537843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95bcbd-cbc0-4aa9-8120-6807ba66f849 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:28.538321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95bcbd-cbc0-4aa9-8120-6807ba66f849 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:28.538810 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95bcbd-cbc0-4aa9-8120-6807ba66f849 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:28.539808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d95bcbd-cbc0-4aa9-8120-6807ba66f849 None admin] RBAC: Adding query filter params (role.id=144c8425688b409384e1e8ed8279d88e, scope.project.id=34ca99f1da904d608e324f65f6571250, user.id=3d958849e02b4ae8bcb375e7da0949e2) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:28.540373 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d95bcbd-cbc0-4aa9-8120-6807ba66f849 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:28.542603 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d95bcbd-cbc0-4aa9-8120-6807ba66f849 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:28.555843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 105/204] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:17:28 2022] GET /identity/v3/role_assignments?user.id=3d958849e02b4ae8bcb375e7da0949e2&scope.project.id=34ca99f1da904d608e324f65f6571250&role.id=144c8425688b409384e1e8ed8279d88e => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:28.557833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95bcbd-cbc0-4aa9-8120-6807ba66f849 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:28.571614 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:28.571949 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa5e35-bdf5-439a-bbf9-f9e8738ddef2 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:28.597862 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:28.619136 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509a198-5e6b-475c-a99e-f2abcca60537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:28.621678 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509a198-5e6b-475c-a99e-f2abcca60537 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:28.623853 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509a198-5e6b-475c-a99e-f2abcca60537 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:28.624345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509a198-5e6b-475c-a99e-f2abcca60537 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:28.715066 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509a198-5e6b-475c-a99e-f2abcca60537 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:28.722733 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509a198-5e6b-475c-a99e-f2abcca60537 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:28.780247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95bcbd-cbc0-4aa9-8120-6807ba66f849 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:28.856275 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 100/205] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:28.861318 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509a198-5e6b-475c-a99e-f2abcca60537 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:28.875689 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d384a-2749-488a-af51-70b1fc099ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:28.876545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d384a-2749-488a-af51-70b1fc099ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:28.876838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d384a-2749-488a-af51-70b1fc099ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:28.877208 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d384a-2749-488a-af51-70b1fc099ba9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:28.878210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5d384a-2749-488a-af51-70b1fc099ba9 None admin] RBAC: Adding query filter params (role.id=47cde553d7ee465bbe685a05c00b94e2, scope.project.id=34ca99f1da904d608e324f65f6571250, user.id=3d958849e02b4ae8bcb375e7da0949e2) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:28.878425 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5d384a-2749-488a-af51-70b1fc099ba9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:28.879946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5d384a-2749-488a-af51-70b1fc099ba9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:28.892697 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:28.902505 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 106/206] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:17:28 2022] GET /identity/v3/role_assignments?user.id=3d958849e02b4ae8bcb375e7da0949e2&scope.project.id=34ca99f1da904d608e324f65f6571250&role.id=47cde553d7ee465bbe685a05c00b94e2 => generated 259 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:28.984595 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509a198-5e6b-475c-a99e-f2abcca60537 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:29.028662 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca51bd6-547f-41ea-b143-302ef7d6234c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:29.029322 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca51bd6-547f-41ea-b143-302ef7d6234c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:29.029604 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca51bd6-547f-41ea-b143-302ef7d6234c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:29.029880 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca51bd6-547f-41ea-b143-302ef7d6234c None admin] PATH_INFO: `/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:29.031090 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca51bd6-547f-41ea-b143-302ef7d6234c None admin] RBAC: Authorizing `identity:get_project(project_id=5a84e3fab8334ecca0cd4aae83cb5317)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:29.032616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca51bd6-547f-41ea-b143-302ef7d6234c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:29.034332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 101/207] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:28 2022] GET /identity/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:29.040190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d384a-2749-488a-af51-70b1fc099ba9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:29.126336 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:29.164910 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d384a-2749-488a-af51-70b1fc099ba9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:29.215499 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656475e1-aacf-41f7-95d6-4b020ae7151d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:29.216134 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656475e1-aacf-41f7-95d6-4b020ae7151d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:29.216439 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656475e1-aacf-41f7-95d6-4b020ae7151d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:29.216716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656475e1-aacf-41f7-95d6-4b020ae7151d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:29.217522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656475e1-aacf-41f7-95d6-4b020ae7151d None admin] RBAC: Adding query filter params (role.id=7f46f07e73294f919cb56f7a97b324d7, scope.project.id=5a84e3fab8334ecca0cd4aae83cb5317, user.id=f8d8e060c5cd456fa2c4a58592080111) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:29.217830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656475e1-aacf-41f7-95d6-4b020ae7151d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:29.219273 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656475e1-aacf-41f7-95d6-4b020ae7151d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:29.229193 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 107/208] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:17:29 2022] GET /identity/v3/role_assignments?user.id=f8d8e060c5cd456fa2c4a58592080111&scope.project.id=5a84e3fab8334ecca0cd4aae83cb5317&role.id=7f46f07e73294f919cb56f7a97b324d7 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:29.753457 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b7024-0493-4494-a352-09956956e647 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:29.754004 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b7024-0493-4494-a352-09956956e647 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:29.754004 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b7024-0493-4494-a352-09956956e647 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:29.755579 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 102/209] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:29.761114 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be539b-0975-4162-b1d1-c528918c4698 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:29.761516 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be539b-0975-4162-b1d1-c528918c4698 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:29.761688 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be539b-0975-4162-b1d1-c528918c4698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:29.781050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e3be539b-0975-4162-b1d1-c528918c4698 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:29.782237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e3be539b-0975-4162-b1d1-c528918c4698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:29.790076 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143604de-42cb-481b-acb9-0d12e0fcfc36 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:29.790362 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143604de-42cb-481b-acb9-0d12e0fcfc36 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:29.790638 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143604de-42cb-481b-acb9-0d12e0fcfc36 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:29.792064 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 103/210] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:17:29.797201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f7190f-d4f6-42d0-99c0-f67691d67ca1 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:29.797489 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f7190f-d4f6-42d0-99c0-f67691d67ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:29.797765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f7190f-d4f6-42d0-99c0-f67691d67ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:29.817574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-38f7190f-d4f6-42d0-99c0-f67691d67ca1 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:29.818814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-38f7190f-d4f6-42d0-99c0-f67691d67ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:29.863592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 108/211] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:29.887215 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be539b-0975-4162-b1d1-c528918c4698 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:29.905198 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 104/212] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 12:17:29.929384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f7190f-d4f6-42d0-99c0-f67691d67ca1 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:29.952854 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:29.960431 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:30.005250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be539b-0975-4162-b1d1-c528918c4698 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:30.041821 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075fe98-af50-46e4-a57a-e8999a83a6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:30.042548 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075fe98-af50-46e4-a57a-e8999a83a6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:30.042920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075fe98-af50-46e4-a57a-e8999a83a6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:30.043269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075fe98-af50-46e4-a57a-e8999a83a6e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:30.050888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f7190f-d4f6-42d0-99c0-f67691d67ca1 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:30.054409 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075fe98-af50-46e4-a57a-e8999a83a6e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:30.060388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-2075fe98-af50-46e4-a57a-e8999a83a6e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:30.060388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:30.060388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:30.060388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:30.060388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:30.060388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:30.060388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:30.060388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:30.060388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:30.060388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:30.060388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:30.060388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:30.060388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:30.060388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:30.060388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:30.060388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:30.061255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:30.061255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:30.061255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:30.061255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:30.061255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:30.061255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:30.061255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:30.061255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:30.061255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:30.061255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:30.061255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:30.061255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:30.061255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:30.061255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:30.061255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:30.061255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:30.062183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:30.062183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:30.062183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:30.062183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:30.062183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:30.062183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:30.062183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:30.062183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:30.062183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:30.062183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:30.062183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:30.062183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:30.062183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 109/213] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:30.068665 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075fe98-af50-46e4-a57a-e8999a83a6e3 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:30.090073 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c97d7-1f61-488c-9a45-e8e1bc53867e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:30.090710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c97d7-1f61-488c-9a45-e8e1bc53867e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:30.090991 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c97d7-1f61-488c-9a45-e8e1bc53867e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:30.091267 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c97d7-1f61-488c-9a45-e8e1bc53867e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:30.092452 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c97d7-1f61-488c-9a45-e8e1bc53867e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:30.093956 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c97d7-1f61-488c-9a45-e8e1bc53867e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:30.096032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 105/214] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:17:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:30.105174 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c97d7-1f61-488c-9a45-e8e1bc53867e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:30.177625 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075fe98-af50-46e4-a57a-e8999a83a6e3 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:30.202429 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:30.214586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1db8c-c283-495c-b80d-8c803f70a701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:30.215206 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1db8c-c283-495c-b80d-8c803f70a701 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:30.215490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1db8c-c283-495c-b80d-8c803f70a701 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:30.215768 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1db8c-c283-495c-b80d-8c803f70a701 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:30.216512 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1db8c-c283-495c-b80d-8c803f70a701 None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:30.216816 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1db8c-c283-495c-b80d-8c803f70a701 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:30.218189 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1db8c-c283-495c-b80d-8c803f70a701 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:30.224621 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:30.226885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 110/215] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:30.233062 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1db8c-c283-495c-b80d-8c803f70a701 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:30.242073 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c97d7-1f61-488c-9a45-e8e1bc53867e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:30.287423 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f83d4-f3f2-4106-8a76-6cf8f3000b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:30.288831 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f83d4-f3f2-4106-8a76-6cf8f3000b2f None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:30.289409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f83d4-f3f2-4106-8a76-6cf8f3000b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:30.289979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f83d4-f3f2-4106-8a76-6cf8f3000b2f None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:30.291137 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9f83d4-f3f2-4106-8a76-6cf8f3000b2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:30.294043 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9f83d4-f3f2-4106-8a76-6cf8f3000b2f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:30.325656 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 106/216] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Aug 6 12:17:30 2022] POST /identity/v3/users => generated 297 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:17:30.365095 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1db8c-c283-495c-b80d-8c803f70a701 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:30.406782 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf8c55-f3fc-4464-ab26-8e9ceeaf9705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:30.407456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf8c55-f3fc-4464-ab26-8e9ceeaf9705 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:30.407742 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf8c55-f3fc-4464-ab26-8e9ceeaf9705 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:30.408015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf8c55-f3fc-4464-ab26-8e9ceeaf9705 None admin] PATH_INFO: `/v3/users/6359d9cca28849ce9097bd1e7a4928ab` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:30.426756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf8c55-f3fc-4464-ab26-8e9ceeaf9705 None admin] RBAC: Authorizing `identity:get_user(user_id=6359d9cca28849ce9097bd1e7a4928ab)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:30.428251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf8c55-f3fc-4464-ab26-8e9ceeaf9705 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:30.430384 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 111/217] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:30 2022] GET /identity/v3/users/6359d9cca28849ce9097bd1e7a4928ab => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:30.437957 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f83d4-f3f2-4106-8a76-6cf8f3000b2f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:30.549638 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f83d4-f3f2-4106-8a76-6cf8f3000b2f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:30.587344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d5cb4-f1c4-434e-b8f1-5d4e6a0f4927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:30.587973 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d5cb4-f1c4-434e-b8f1-5d4e6a0f4927 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:30.588822 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d5cb4-f1c4-434e-b8f1-5d4e6a0f4927 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:30.589099 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d5cb4-f1c4-434e-b8f1-5d4e6a0f4927 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:30.604668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:30.667619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d5cb4-f1c4-434e-b8f1-5d4e6a0f4927 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:30.669235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d5cb4-f1c4-434e-b8f1-5d4e6a0f4927 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:30.743025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d527228-c554-49a7-84b8-ee4112b28e3f None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:30.743320 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d527228-c554-49a7-84b8-ee4112b28e3f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:30.743584 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d527228-c554-49a7-84b8-ee4112b28e3f None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:30.746912 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 112/218] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:30.752177 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef95d8-a1f7-4023-8b1c-2ceb2a0acbd3 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:30.752507 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef95d8-a1f7-4023-8b1c-2ceb2a0acbd3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:30.752789 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef95d8-a1f7-4023-8b1c-2ceb2a0acbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:30.778127 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-09ef95d8-a1f7-4023-8b1c-2ceb2a0acbd3 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:30.779324 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-09ef95d8-a1f7-4023-8b1c-2ceb2a0acbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:30.789128 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 107/219] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:30.795849 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d5cb4-f1c4-434e-b8f1-5d4e6a0f4927 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:30.850238 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:30.877282 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:30.909380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 113/220] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 12:17:30.948883 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d5cb4-f1c4-434e-b8f1-5d4e6a0f4927 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:30.975882 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef95d8-a1f7-4023-8b1c-2ceb2a0acbd3 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:31.008328 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2506a-0bcc-4ce2-845b-9a231eb52cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:31.008328 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2506a-0bcc-4ce2-845b-9a231eb52cfd None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:31.008328 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2506a-0bcc-4ce2-845b-9a231eb52cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:31.008328 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2506a-0bcc-4ce2-845b-9a231eb52cfd None admin] PATH_INFO: `/v3/projects/34ca99f1da904d608e324f65f6571250` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:31.008715 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2506a-0bcc-4ce2-845b-9a231eb52cfd None admin] RBAC: Authorizing `identity:get_project(project_id=34ca99f1da904d608e324f65f6571250)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:31.008715 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2506a-0bcc-4ce2-845b-9a231eb52cfd None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:31.012895 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 108/221] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:30 2022] GET /identity/v3/projects/34ca99f1da904d608e324f65f6571250 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:31.018341 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2506a-0bcc-4ce2-845b-9a231eb52cfd None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:31.145609 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2506a-0bcc-4ce2-845b-9a231eb52cfd None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:31.160899 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:31.193183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef95d8-a1f7-4023-8b1c-2ceb2a0acbd3 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:31.194041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0258e75-4026-4e2e-bb76-1a82072428be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:31.194663 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0258e75-4026-4e2e-bb76-1a82072428be None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:31.194943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0258e75-4026-4e2e-bb76-1a82072428be None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:31.195216 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0258e75-4026-4e2e-bb76-1a82072428be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:31.196013 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0258e75-4026-4e2e-bb76-1a82072428be None admin] RBAC: Adding query filter params (role.id=0fd7b75299b64c54971d9bfeaec30f4e, scope.project.id=34ca99f1da904d608e324f65f6571250, user.id=6359d9cca28849ce9097bd1e7a4928ab) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:31.196358 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0258e75-4026-4e2e-bb76-1a82072428be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:31.197961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0258e75-4026-4e2e-bb76-1a82072428be None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:31.212388 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 109/222] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:17:31 2022] GET /identity/v3/role_assignments?user.id=6359d9cca28849ce9097bd1e7a4928ab&scope.project.id=34ca99f1da904d608e324f65f6571250&role.id=0fd7b75299b64c54971d9bfeaec30f4e => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:31.248776 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f409a1-24cd-4f3d-b283-28c03818a7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:31.250198 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f409a1-24cd-4f3d-b283-28c03818a7cf None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:31.250904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f409a1-24cd-4f3d-b283-28c03818a7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:31.251586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f409a1-24cd-4f3d-b283-28c03818a7cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:31.264481 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f409a1-24cd-4f3d-b283-28c03818a7cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:31.270845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-a8f409a1-24cd-4f3d-b283-28c03818a7cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:17:31.270845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:31.270845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:31.270845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:31.270845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:31.270845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:31.270845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:31.270845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:31.270845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:31.270845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:31.270845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:31.270845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:31.270845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:31.270845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:31.270845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:31.270845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:31.272611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:31.272611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:31.272611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:31.272611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:31.272611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:31.272611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:31.272611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:31.272611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:31.272611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:31.272611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:31.272611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:31.272611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:31.272611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:31.272611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:31.272611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:31.272611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:31.274192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:31.274192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:31.274192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:31.274192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:31.274192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:31.274192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:31.274192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:31.274192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:31.274192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:31.274192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:31.274192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:17:31.274192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:31.278662 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 114/223] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:17:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:31.286235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0258e75-4026-4e2e-bb76-1a82072428be None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:31.367099 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01cb6e-3b1f-4308-9a0d-2e282f774e5c None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:31.367375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01cb6e-3b1f-4308-9a0d-2e282f774e5c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:31.367659 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01cb6e-3b1f-4308-9a0d-2e282f774e5c None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:31.369377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 115/224] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:17:31.378529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d018e5e-2dcf-4ba3-ba74-3c490df263e2 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:31.378809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d018e5e-2dcf-4ba3-ba74-3c490df263e2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:31.380824 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d018e5e-2dcf-4ba3-ba74-3c490df263e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:31.416557 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9d018e5e-2dcf-4ba3-ba74-3c490df263e2 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:31.417756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9d018e5e-2dcf-4ba3-ba74-3c490df263e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:31.447583 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0258e75-4026-4e2e-bb76-1a82072428be None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:31.484015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c889b062-37ed-4366-88f0-784fdca1faa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:31.484660 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889b062-37ed-4366-88f0-784fdca1faa1 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:31.484938 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889b062-37ed-4366-88f0-784fdca1faa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:31.485208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889b062-37ed-4366-88f0-784fdca1faa1 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:31.485399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:31.486304 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c889b062-37ed-4366-88f0-784fdca1faa1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:31.486630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c889b062-37ed-4366-88f0-784fdca1faa1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:31.488256 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c889b062-37ed-4366-88f0-784fdca1faa1 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:31.498523 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 110/225] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:31.504691 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c889b062-37ed-4366-88f0-784fdca1faa1 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:31.570095 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 116/226] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:31.575218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:31.593554 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d018e5e-2dcf-4ba3-ba74-3c490df263e2 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:31.703574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c889b062-37ed-4366-88f0-784fdca1faa1 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:31.766417 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d018e5e-2dcf-4ba3-ba74-3c490df263e2 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:31.769591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41407ba-7740-45a3-b96b-7a8960f02c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:31.770258 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41407ba-7740-45a3-b96b-7a8960f02c29 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:31.770545 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41407ba-7740-45a3-b96b-7a8960f02c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:31.770827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41407ba-7740-45a3-b96b-7a8960f02c29 None admin] PATH_INFO: `/v3/users/3d958849e02b4ae8bcb375e7da0949e2` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:31.772025 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41407ba-7740-45a3-b96b-7a8960f02c29 None admin] RBAC: Authorizing `identity:get_user(user_id=3d958849e02b4ae8bcb375e7da0949e2)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:31.773530 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41407ba-7740-45a3-b96b-7a8960f02c29 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:31.784669 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 111/227] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:31 2022] GET /identity/v3/users/3d958849e02b4ae8bcb375e7da0949e2 => generated 283 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:31.785542 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41407ba-7740-45a3-b96b-7a8960f02c29 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:31.809042 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc0438-93cd-4e82-af87-b93bf0336624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:31.810414 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc0438-93cd-4e82-af87-b93bf0336624 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:31.810891 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc0438-93cd-4e82-af87-b93bf0336624 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:31.811358 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc0438-93cd-4e82-af87-b93bf0336624 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:31.830248 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc0438-93cd-4e82-af87-b93bf0336624 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:31.840178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-34fc0438-93cd-4e82-af87-b93bf0336624 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:17:31.840178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:31.840178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:31.840178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:31.840178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:31.840178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:31.840178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:31.840178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:31.840178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:31.840178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:31.840178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:31.840178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:31.840178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:31.840178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:31.840178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:31.840178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:31.841076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:31.841076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:31.841076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:31.841076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:31.841076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:31.841076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:31.841076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:31.841076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:31.841076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:31.841076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:31.841076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:31.841076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:31.841076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:31.841076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:31.841076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:31.841076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:31.841861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:31.841861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:31.841861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:31.841861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:31.841861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:31.841861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:31.841861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:31.841861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:31.841861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:31.841861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:31.841861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:17:31.841861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:31.844956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 117/228] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Aug 6 12:17:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:17:31.852665 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc0438-93cd-4e82-af87-b93bf0336624 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:31.905136 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:31.984691 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41407ba-7740-45a3-b96b-7a8960f02c29 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:32.007524 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:32.043103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc0438-93cd-4e82-af87-b93bf0336624 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:32.052871 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f042c42f-864c-45d3-b6bc-be2905f4f71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:32.053524 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042c42f-864c-45d3-b6bc-be2905f4f71a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:32.053799 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042c42f-864c-45d3-b6bc-be2905f4f71a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:32.054072 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f042c42f-864c-45d3-b6bc-be2905f4f71a None admin] PATH_INFO: `/v3/projects/34ca99f1da904d608e324f65f6571250` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:32.056214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f042c42f-864c-45d3-b6bc-be2905f4f71a None admin] RBAC: Authorizing `identity:get_project(project_id=34ca99f1da904d608e324f65f6571250)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:32.058208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f042c42f-864c-45d3-b6bc-be2905f4f71a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:32.062092 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 112/229] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:31 2022] GET /identity/v3/projects/34ca99f1da904d608e324f65f6571250 => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:32.065664 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f042c42f-864c-45d3-b6bc-be2905f4f71a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:32.108794 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57f4e7-013d-4aaa-a6f1-4eee08963989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:32.109413 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57f4e7-013d-4aaa-a6f1-4eee08963989 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:32.109687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57f4e7-013d-4aaa-a6f1-4eee08963989 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:32.109953 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57f4e7-013d-4aaa-a6f1-4eee08963989 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:32.110666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57f4e7-013d-4aaa-a6f1-4eee08963989 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:32.110965 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57f4e7-013d-4aaa-a6f1-4eee08963989 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:32.112340 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57f4e7-013d-4aaa-a6f1-4eee08963989 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:32.132177 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 118/230] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:17:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:32.137672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57f4e7-013d-4aaa-a6f1-4eee08963989 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:32.210059 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f042c42f-864c-45d3-b6bc-be2905f4f71a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:32.251840 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e105273-fa96-4c80-899a-22b229ead558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:32.252502 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e105273-fa96-4c80-899a-22b229ead558 None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:32.252777 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e105273-fa96-4c80-899a-22b229ead558 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:32.253048 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e105273-fa96-4c80-899a-22b229ead558 None admin] PATH_INFO: `/v3/projects/34ca99f1da904d608e324f65f6571250/users/3d958849e02b4ae8bcb375e7da0949e2/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:32.260603 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e105273-fa96-4c80-899a-22b229ead558 None admin] RBAC: Authorizing `identity:create_grant(project_id=34ca99f1da904d608e324f65f6571250, user_id=3d958849e02b4ae8bcb375e7da0949e2, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:32.262061 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e105273-fa96-4c80-899a-22b229ead558 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:32.287684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 113/231] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:17:32 2022] PUT /identity/v3/projects/34ca99f1da904d608e324f65f6571250/users/3d958849e02b4ae8bcb375e7da0949e2/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:17:32.332905 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:32.354564 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57f4e7-013d-4aaa-a6f1-4eee08963989 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:32.406946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33095d-9bb4-4a49-bd48-a59c6e5655fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:32.407606 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33095d-9bb4-4a49-bd48-a59c6e5655fc None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:32.407881 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33095d-9bb4-4a49-bd48-a59c6e5655fc None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:32.408152 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33095d-9bb4-4a49-bd48-a59c6e5655fc None admin] PATH_INFO: `/v3/users/3d958849e02b4ae8bcb375e7da0949e2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:32.409435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33095d-9bb4-4a49-bd48-a59c6e5655fc None admin] RBAC: Authorizing `identity:get_user(user_id=3d958849e02b4ae8bcb375e7da0949e2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:32.410972 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33095d-9bb4-4a49-bd48-a59c6e5655fc None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:32.412736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 119/232] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:32 2022] GET /identity/v3/users/3d958849e02b4ae8bcb375e7da0949e2 => generated 283 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:32.419102 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e105273-fa96-4c80-899a-22b229ead558 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:32.468563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:32.529238 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e105273-fa96-4c80-899a-22b229ead558 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:32.565751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c5184-9479-49eb-af98-026508ec61cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:32.566386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c5184-9479-49eb-af98-026508ec61cf None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:32.566663 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c5184-9479-49eb-af98-026508ec61cf None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:32.566933 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c5184-9479-49eb-af98-026508ec61cf None admin] PATH_INFO: `/v3/projects/34ca99f1da904d608e324f65f6571250` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:32.568092 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c5184-9479-49eb-af98-026508ec61cf None admin] RBAC: Authorizing `identity:get_project(project_id=34ca99f1da904d608e324f65f6571250)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:32.569589 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c5184-9479-49eb-af98-026508ec61cf None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:32.576525 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 114/233] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:32 2022] GET /identity/v3/projects/34ca99f1da904d608e324f65f6571250 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:32.577895 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33095d-9bb4-4a49-bd48-a59c6e5655fc None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:32.690016 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33095d-9bb4-4a49-bd48-a59c6e5655fc None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:32.724733 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e2712-a7db-41ba-ba21-664c3a8b3a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:32.725398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e2712-a7db-41ba-ba21-664c3a8b3a94 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:32.725675 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e2712-a7db-41ba-ba21-664c3a8b3a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:32.725949 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e2712-a7db-41ba-ba21-664c3a8b3a94 None admin] PATH_INFO: `/v3/projects/34ca99f1da904d608e324f65f6571250/users/3d958849e02b4ae8bcb375e7da0949e2/roles/47cde553d7ee465bbe685a05c00b94e2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:32.728208 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e2712-a7db-41ba-ba21-664c3a8b3a94 None admin] RBAC: Authorizing `identity:create_grant(project_id=34ca99f1da904d608e324f65f6571250, user_id=3d958849e02b4ae8bcb375e7da0949e2, role_id=47cde553d7ee465bbe685a05c00b94e2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:32.730278 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e2712-a7db-41ba-ba21-664c3a8b3a94 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:32.749815 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 120/234] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:17:32 2022] PUT /identity/v3/projects/34ca99f1da904d608e324f65f6571250/users/3d958849e02b4ae8bcb375e7da0949e2/roles/47cde553d7ee465bbe685a05c00b94e2 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:17:32.904548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27937f-92d5-4494-b359-d8edd474487e None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:32.904991 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27937f-92d5-4494-b359-d8edd474487e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:32.905212 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27937f-92d5-4494-b359-d8edd474487e None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:32.906496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 115/235] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:32.913611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157ddc7-6bcd-4730-85ba-922780c78186 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:32.914333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157ddc7-6bcd-4730-85ba-922780c78186 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:32.914333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157ddc7-6bcd-4730-85ba-922780c78186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:32.935950 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e157ddc7-6bcd-4730-85ba-922780c78186 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:32.937273 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e157ddc7-6bcd-4730-85ba-922780c78186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:32.947865 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:33.031628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ee59d-3511-42f2-a3c3-aa9894ccb325 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:33.032123 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ee59d-3511-42f2-a3c3-aa9894ccb325 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:33.032198 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ee59d-3511-42f2-a3c3-aa9894ccb325 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:33.033883 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 116/236] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:17:33.035086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 121/237] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:33.038666 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d710034-657d-48c5-be8b-ef14e3b0f997 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:33.038947 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d710034-657d-48c5-be8b-ef14e3b0f997 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:33.039220 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d710034-657d-48c5-be8b-ef14e3b0f997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:33.057901 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3d710034-657d-48c5-be8b-ef14e3b0f997 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:33.059077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3d710034-657d-48c5-be8b-ef14e3b0f997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:33.082361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157ddc7-6bcd-4730-85ba-922780c78186 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:33.137349 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 117/238] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:33.221623 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d710034-657d-48c5-be8b-ef14e3b0f997 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:33.227984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157ddc7-6bcd-4730-85ba-922780c78186 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:33.264546 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2fdbc8-20e7-433c-a4ce-a2b78c4ef001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:33.265199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2fdbc8-20e7-433c-a4ce-a2b78c4ef001 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:33.265477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2fdbc8-20e7-433c-a4ce-a2b78c4ef001 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:33.265751 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2fdbc8-20e7-433c-a4ce-a2b78c4ef001 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:33.267288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2fdbc8-20e7-433c-a4ce-a2b78c4ef001 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:33.268799 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2fdbc8-20e7-433c-a4ce-a2b78c4ef001 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:33.270730 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 122/239] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:17:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:33.277384 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2fdbc8-20e7-433c-a4ce-a2b78c4ef001 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:33.295063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:33.312659 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:33.422359 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d710034-657d-48c5-be8b-ef14e3b0f997 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:33.465596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2fdbc8-20e7-433c-a4ce-a2b78c4ef001 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:33.483003 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90852a9d-7d58-4e0f-b5a1-91675930021b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:33.483568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90852a9d-7d58-4e0f-b5a1-91675930021b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:33.483845 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90852a9d-7d58-4e0f-b5a1-91675930021b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:33.484117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90852a9d-7d58-4e0f-b5a1-91675930021b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:33.505905 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90852a9d-7d58-4e0f-b5a1-91675930021b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:33.511578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-90852a9d-7d58-4e0f-b5a1-91675930021b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 12:17:33.511578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:33.511578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:33.511578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:33.511578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:33.511578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:33.511578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:33.511578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:33.511578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:33.511578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:33.511578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:33.511578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:33.511578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:33.511578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:33.511578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:33.511578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:33.512554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:33.512554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:33.512554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:33.512554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:33.512554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:33.512554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:33.512554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:33.512554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:33.512554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:33.512554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:33.512554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:33.512554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:33.512554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:33.512554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:33.512554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:33.512554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:33.513339 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:33.513339 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:33.513339 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:33.513339 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:33.513339 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:33.513339 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:33.513339 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:33.513339 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:33.513339 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:33.513339 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:33.513339 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 12:17:33.513339 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:17:33.513938 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 118/240] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:17:33.520394 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90852a9d-7d58-4e0f-b5a1-91675930021b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:33.529139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d283128-9875-412a-a0d5-6aae914f856e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:33.529761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d283128-9875-412a-a0d5-6aae914f856e None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:33.530034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d283128-9875-412a-a0d5-6aae914f856e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:33.530304 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d283128-9875-412a-a0d5-6aae914f856e None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:33.530980 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d283128-9875-412a-a0d5-6aae914f856e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:33.532414 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d283128-9875-412a-a0d5-6aae914f856e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:33.571139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 123/241] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Aug 6 12:17:33 2022] POST /identity/v3/users => generated 293 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:17:33.674527 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:33.684909 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90852a9d-7d58-4e0f-b5a1-91675930021b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:33.725832 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba18f08-634c-4dd2-a34a-e6fba22c1a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:33.726449 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba18f08-634c-4dd2-a34a-e6fba22c1a90 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:33.726761 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba18f08-634c-4dd2-a34a-e6fba22c1a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:33.727044 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba18f08-634c-4dd2-a34a-e6fba22c1a90 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:33.727840 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba18f08-634c-4dd2-a34a-e6fba22c1a90 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:33.728150 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba18f08-634c-4dd2-a34a-e6fba22c1a90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:33.729694 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba18f08-634c-4dd2-a34a-e6fba22c1a90 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:33.736070 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7daf9-76e8-46b0-86a7-134a8c69e950 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:33.736402 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7daf9-76e8-46b0-86a7-134a8c69e950 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:33.736561 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7daf9-76e8-46b0-86a7-134a8c69e950 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:33.738332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 124/242] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:33.744773 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e2da2-ae9e-49c5-95ba-22343da4dbcf None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:33.745067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e2da2-ae9e-49c5-95ba-22343da4dbcf None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:33.745323 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e2da2-ae9e-49c5-95ba-22343da4dbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:33.755152 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 119/243] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:33.760439 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba18f08-634c-4dd2-a34a-e6fba22c1a90 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:33.776968 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-232e2da2-ae9e-49c5-95ba-22343da4dbcf None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:33.779928 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-232e2da2-ae9e-49c5-95ba-22343da4dbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:33.896021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:33.907143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 125/244] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:33.928956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e2da2-ae9e-49c5-95ba-22343da4dbcf None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:33.931329 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba18f08-634c-4dd2-a34a-e6fba22c1a90 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:33.968153 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72a1af-83ea-4b55-9d6f-159e1b7b8eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:33.968841 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72a1af-83ea-4b55-9d6f-159e1b7b8eca None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:33.969117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72a1af-83ea-4b55-9d6f-159e1b7b8eca None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:33.969388 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72a1af-83ea-4b55-9d6f-159e1b7b8eca None admin] PATH_INFO: `/v3/users/5c72c302441649d3a54d8272cbdc2543` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:33.985738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad72a1af-83ea-4b55-9d6f-159e1b7b8eca None admin] RBAC: Authorizing `identity:get_user(user_id=5c72c302441649d3a54d8272cbdc2543)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:33.987412 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad72a1af-83ea-4b55-9d6f-159e1b7b8eca None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:34.001138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 120/245] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:33 2022] GET /identity/v3/users/5c72c302441649d3a54d8272cbdc2543 => generated 297 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:34.007680 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72a1af-83ea-4b55-9d6f-159e1b7b8eca None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:34.038376 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:34.057680 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e2da2-ae9e-49c5-95ba-22343da4dbcf None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:34.105010 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569dad94-3a2c-4823-8f20-e00a4eb657c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:34.105010 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569dad94-3a2c-4823-8f20-e00a4eb657c1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:34.105268 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569dad94-3a2c-4823-8f20-e00a4eb657c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:34.105337 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569dad94-3a2c-4823-8f20-e00a4eb657c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:34.120510 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569dad94-3a2c-4823-8f20-e00a4eb657c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:34.127284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-569dad94-3a2c-4823-8f20-e00a4eb657c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:34.127284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:34.127284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:34.127284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:34.127284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:34.127284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:34.127284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:34.127284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:34.127284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:34.127284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:34.127284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:34.127284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:34.127284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:34.127284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:34.127284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:34.127284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:34.128210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:34.128210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:34.128210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:34.128210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:34.128210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:34.128210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:34.128210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:34.128210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:34.128210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:34.128210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:34.128210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:34.128210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:34.128210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:34.128210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:34.128210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:34.128210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:34.129153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:34.129153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:34.129153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:34.129153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:34.129153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:34.129153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:34.129153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:34.129153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:34.129153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:34.129153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:34.129153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:34.129153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:34.129153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 126/246] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:17:34.137599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569dad94-3a2c-4823-8f20-e00a4eb657c1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:34.161525 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72a1af-83ea-4b55-9d6f-159e1b7b8eca None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:34.201480 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad84f9-4d12-4eb8-8417-e83c1f1b25bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:34.202492 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad84f9-4d12-4eb8-8417-e83c1f1b25bc None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:34.202952 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad84f9-4d12-4eb8-8417-e83c1f1b25bc None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:34.203401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad84f9-4d12-4eb8-8417-e83c1f1b25bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:34.218934 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:34.253630 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569dad94-3a2c-4823-8f20-e00a4eb657c1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:34.278462 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad84f9-4d12-4eb8-8417-e83c1f1b25bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:34.280189 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad84f9-4d12-4eb8-8417-e83c1f1b25bc None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:34.289899 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc5901-2f9f-4f0c-ba13-797e49bceec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:34.290609 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc5901-2f9f-4f0c-ba13-797e49bceec9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:34.290956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc5901-2f9f-4f0c-ba13-797e49bceec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:34.291375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc5901-2f9f-4f0c-ba13-797e49bceec9 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:34.292163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc5901-2f9f-4f0c-ba13-797e49bceec9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:34.292555 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc5901-2f9f-4f0c-ba13-797e49bceec9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:34.293991 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc5901-2f9f-4f0c-ba13-797e49bceec9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:34.296667 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:34.304379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 127/247] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:34.310826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc5901-2f9f-4f0c-ba13-797e49bceec9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:34.393941 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 121/248] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:34.401946 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad84f9-4d12-4eb8-8417-e83c1f1b25bc None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:34.417863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc5901-2f9f-4f0c-ba13-797e49bceec9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:34.462093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:34.483929 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d9323-8d02-4112-a589-5e2c422522aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:34.484606 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d9323-8d02-4112-a589-5e2c422522aa None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:34.484934 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d9323-8d02-4112-a589-5e2c422522aa None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:34.485206 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d9323-8d02-4112-a589-5e2c422522aa None admin] PATH_INFO: `/v3/users/6359d9cca28849ce9097bd1e7a4928ab` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:34.488155 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d9323-8d02-4112-a589-5e2c422522aa None admin] RBAC: Authorizing `identity:get_user(user_id=6359d9cca28849ce9097bd1e7a4928ab)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:34.489644 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71d9323-8d02-4112-a589-5e2c422522aa None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:34.500137 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 128/249] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:34 2022] GET /identity/v3/users/6359d9cca28849ce9097bd1e7a4928ab => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:34.501818 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d9323-8d02-4112-a589-5e2c422522aa None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:34.608566 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad84f9-4d12-4eb8-8417-e83c1f1b25bc None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:34.640691 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:34.686097 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4c7a3-9dd4-496d-a670-d379855eb461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:34.686748 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4c7a3-9dd4-496d-a670-d379855eb461 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:34.687016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4c7a3-9dd4-496d-a670-d379855eb461 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:34.687290 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4c7a3-9dd4-496d-a670-d379855eb461 None admin] PATH_INFO: `/v3/projects/34ca99f1da904d608e324f65f6571250` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:34.691576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4c7a3-9dd4-496d-a670-d379855eb461 None admin] RBAC: Authorizing `identity:get_project(project_id=34ca99f1da904d608e324f65f6571250)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:34.695004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71d9323-8d02-4112-a589-5e2c422522aa None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:34.697581 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4c7a3-9dd4-496d-a670-d379855eb461 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:34.699363 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 122/250] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:34 2022] GET /identity/v3/projects/34ca99f1da904d608e324f65f6571250 => generated 302 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:34.709690 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4c7a3-9dd4-496d-a670-d379855eb461 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:34.763854 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb2028-0019-41e2-b03e-1402ab1e0f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:34.765133 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb2028-0019-41e2-b03e-1402ab1e0f1b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:34.765415 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb2028-0019-41e2-b03e-1402ab1e0f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:34.765684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb2028-0019-41e2-b03e-1402ab1e0f1b None admin] PATH_INFO: `/v3/projects/34ca99f1da904d608e324f65f6571250` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:34.769034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fb2028-0019-41e2-b03e-1402ab1e0f1b None admin] RBAC: Authorizing `identity:get_project(project_id=34ca99f1da904d608e324f65f6571250)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:34.770476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fb2028-0019-41e2-b03e-1402ab1e0f1b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:34.772143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 129/251] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:34 2022] GET /identity/v3/projects/34ca99f1da904d608e324f65f6571250 => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:34.786085 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb2028-0019-41e2-b03e-1402ab1e0f1b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:34.917469 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:34.929610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb2028-0019-41e2-b03e-1402ab1e0f1b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:34.950266 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4c7a3-9dd4-496d-a670-d379855eb461 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:34.967478 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293341be-c579-4cdf-a3d8-2763edb8f222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:34.968154 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293341be-c579-4cdf-a3d8-2763edb8f222 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:34.968466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293341be-c579-4cdf-a3d8-2763edb8f222 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:34.968738 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293341be-c579-4cdf-a3d8-2763edb8f222 None admin] PATH_INFO: `/v3/projects/34ca99f1da904d608e324f65f6571250/users/6359d9cca28849ce9097bd1e7a4928ab/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:34.971026 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293341be-c579-4cdf-a3d8-2763edb8f222 None admin] RBAC: Authorizing `identity:create_grant(project_id=34ca99f1da904d608e324f65f6571250, user_id=6359d9cca28849ce9097bd1e7a4928ab, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:34.972493 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293341be-c579-4cdf-a3d8-2763edb8f222 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:34.986902 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 130/252] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:17:34 2022] PUT /identity/v3/projects/34ca99f1da904d608e324f65f6571250/users/6359d9cca28849ce9097bd1e7a4928ab/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:17:35.043028 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e01a3-4d7b-4136-a35a-c365c6355626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:35.045196 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e01a3-4d7b-4136-a35a-c365c6355626 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:35.045890 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e01a3-4d7b-4136-a35a-c365c6355626 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:35.046625 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e01a3-4d7b-4136-a35a-c365c6355626 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:35.048062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e01a3-4d7b-4136-a35a-c365c6355626 None admin] RBAC: Adding query filter params (role.id=7f46f07e73294f919cb56f7a97b324d7, scope.project.id=34ca99f1da904d608e324f65f6571250, user.id=5c72c302441649d3a54d8272cbdc2543) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:35.048665 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e01a3-4d7b-4136-a35a-c365c6355626 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:35.050748 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e01a3-4d7b-4136-a35a-c365c6355626 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:35.062398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 123/253] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:17:34 2022] GET /identity/v3/role_assignments?user.id=5c72c302441649d3a54d8272cbdc2543&scope.project.id=34ca99f1da904d608e324f65f6571250&role.id=7f46f07e73294f919cb56f7a97b324d7 => generated 259 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:35.204470 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061c0da-0603-41d7-a65b-f68164571284 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:35.204840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061c0da-0603-41d7-a65b-f68164571284 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:35.205121 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061c0da-0603-41d7-a65b-f68164571284 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:35.206946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 131/254] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:35.212173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe86f09-bb7e-41ba-b6cc-3bbe69bea938 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:35.212746 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe86f09-bb7e-41ba-b6cc-3bbe69bea938 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:35.213752 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe86f09-bb7e-41ba-b6cc-3bbe69bea938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:35.241651 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ffe86f09-bb7e-41ba-b6cc-3bbe69bea938 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:35.243254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ffe86f09-bb7e-41ba-b6cc-3bbe69bea938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:35.304739 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:35.343610 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 124/255] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:35.380482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061c0da-0603-41d7-a65b-f68164571284 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:35.391462 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f042dd-aedb-4a04-8080-f631631890d1 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:35.393460 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f042dd-aedb-4a04-8080-f631631890d1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:35.393735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f042dd-aedb-4a04-8080-f631631890d1 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:35.395117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 125/256] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:35 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:35.401759 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb507342-dd2d-4e58-abc2-722878d0f15c None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:35.402042 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb507342-dd2d-4e58-abc2-722878d0f15c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:35.402314 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb507342-dd2d-4e58-abc2-722878d0f15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:35.434327 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-fb507342-dd2d-4e58-abc2-722878d0f15c None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:35.435519 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fb507342-dd2d-4e58-abc2-722878d0f15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:35.574710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 126/257] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:35.578315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061c0da-0603-41d7-a65b-f68164571284 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:35.595600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:35.617668 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a90dc-c526-4734-a2c4-a750b4a288f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:35.618417 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a90dc-c526-4734-a2c4-a750b4a288f4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:35.618705 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a90dc-c526-4734-a2c4-a750b4a288f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:35.618990 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a90dc-c526-4734-a2c4-a750b4a288f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:35.624962 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb507342-dd2d-4e58-abc2-722878d0f15c None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:35.647449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5a90dc-c526-4734-a2c4-a750b4a288f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:35.655654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-5e5a90dc-c526-4734-a2c4-a750b4a288f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:17:35.655654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:35.655654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:35.655654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:35.655654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:35.655654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:35.655654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:35.655654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:35.655654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:35.655654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:35.655654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:35.655654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:35.655654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:35.655654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:35.655654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:35.655654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:35.656601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:35.656601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:35.656601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:35.656601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:35.656601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:35.656601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:35.656601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:35.656601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:35.656601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:35.656601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:35.656601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:35.656601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:35.656601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:35.656601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:35.656601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:35.656601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:35.657618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:35.657618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:35.657618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:35.657618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:35.657618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:35.657618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:35.657618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:35.657618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:35.657618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:35.657618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:35.657618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:17:35.657618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:35.657618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 132/258] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:17:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:35.665381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a90dc-c526-4734-a2c4-a750b4a288f4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:35.816257 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb507342-dd2d-4e58-abc2-722878d0f15c None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:35.824594 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:35.892738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca32c1ea-16c7-47ec-b166-5a295dc8a559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:35.893943 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a90dc-c526-4734-a2c4-a750b4a288f4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:35.895511 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32c1ea-16c7-47ec-b166-5a295dc8a559 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:35.895795 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32c1ea-16c7-47ec-b166-5a295dc8a559 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:35.896068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32c1ea-16c7-47ec-b166-5a295dc8a559 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:35.918002 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca32c1ea-16c7-47ec-b166-5a295dc8a559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:35.923067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-ca32c1ea-16c7-47ec-b166-5a295dc8a559 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:17:35.923067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:35.923067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:35.923067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:35.923067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:35.923067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:35.923067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:35.923067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:35.923067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:35.923067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:35.923067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:35.923067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:35.923067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:35.923067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:35.923067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:35.923067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:35.923942 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:35.923942 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:35.923942 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:35.923942 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:35.923942 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:35.923942 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:35.923942 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:35.923942 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:35.923942 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:35.923942 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:35.923942 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:35.923942 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:35.923942 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:35.923942 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:35.923942 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:35.923942 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:35.924797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:35.924797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:35.924797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:35.924797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:35.924797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:35.924797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:35.924797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:35.924797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:35.924797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:35.924797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:35.924797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:17:35.924797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:17:35.931277 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 127/259] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Aug 6 12:17:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:17:35.938938 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca32c1ea-16c7-47ec-b166-5a295dc8a559 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:35.940805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63805aa4-2753-4d3c-8a9a-6b3e1908bd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:35.941439 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63805aa4-2753-4d3c-8a9a-6b3e1908bd29 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:35.941715 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63805aa4-2753-4d3c-8a9a-6b3e1908bd29 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:35.941987 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63805aa4-2753-4d3c-8a9a-6b3e1908bd29 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:35.942714 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63805aa4-2753-4d3c-8a9a-6b3e1908bd29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:35.943012 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63805aa4-2753-4d3c-8a9a-6b3e1908bd29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:35.944426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63805aa4-2753-4d3c-8a9a-6b3e1908bd29 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:35.967243 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 133/260] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:35.972271 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63805aa4-2753-4d3c-8a9a-6b3e1908bd29 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:36.089609 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:36.110905 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca32c1ea-16c7-47ec-b166-5a295dc8a559 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:36.157079 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee93a6c-f38f-4bbc-b2b4-1f8c09a69849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:36.157625 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee93a6c-f38f-4bbc-b2b4-1f8c09a69849 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:36.157910 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee93a6c-f38f-4bbc-b2b4-1f8c09a69849 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:36.158209 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee93a6c-f38f-4bbc-b2b4-1f8c09a69849 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:36.158919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee93a6c-f38f-4bbc-b2b4-1f8c09a69849 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:36.159222 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee93a6c-f38f-4bbc-b2b4-1f8c09a69849 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:36.160671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee93a6c-f38f-4bbc-b2b4-1f8c09a69849 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:36.176750 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 128/261] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:17:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:36.182893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee93a6c-f38f-4bbc-b2b4-1f8c09a69849 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:36.193942 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63805aa4-2753-4d3c-8a9a-6b3e1908bd29 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:36.261021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd781ed-9784-447a-9125-817d17e979a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:36.261666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd781ed-9784-447a-9125-817d17e979a8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:36.261948 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd781ed-9784-447a-9125-817d17e979a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:36.262220 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd781ed-9784-447a-9125-817d17e979a8 None admin] PATH_INFO: `/v3/users/3d958849e02b4ae8bcb375e7da0949e2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:36.263433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd781ed-9784-447a-9125-817d17e979a8 None admin] RBAC: Authorizing `identity:get_user(user_id=3d958849e02b4ae8bcb375e7da0949e2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:36.264966 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd781ed-9784-447a-9125-817d17e979a8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:36.266542 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 134/262] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:35 2022] GET /identity/v3/users/3d958849e02b4ae8bcb375e7da0949e2 => generated 283 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:36.271983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd781ed-9784-447a-9125-817d17e979a8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:36.272764 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:36.305538 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee93a6c-f38f-4bbc-b2b4-1f8c09a69849 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:36.347335 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a56f79-cc90-4335-926b-ab43ffe1b0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:36.347979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a56f79-cc90-4335-926b-ab43ffe1b0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:36.348261 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a56f79-cc90-4335-926b-ab43ffe1b0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:36.348578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a56f79-cc90-4335-926b-ab43ffe1b0a4 None admin] PATH_INFO: `/v3/users/3d958849e02b4ae8bcb375e7da0949e2` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:36.351337 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a56f79-cc90-4335-926b-ab43ffe1b0a4 None admin] RBAC: Authorizing `identity:get_user(user_id=3d958849e02b4ae8bcb375e7da0949e2)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:36.353082 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a56f79-cc90-4335-926b-ab43ffe1b0a4 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:36.355432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 129/263] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:36 2022] GET /identity/v3/users/3d958849e02b4ae8bcb375e7da0949e2 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:36.361715 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a56f79-cc90-4335-926b-ab43ffe1b0a4 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:36.513987 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd781ed-9784-447a-9125-817d17e979a8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:36.528660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:36.579861 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:36.593081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11ca7f-d548-4f72-9cac-aedcc083447b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:36.593723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11ca7f-d548-4f72-9cac-aedcc083447b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:36.594012 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11ca7f-d548-4f72-9cac-aedcc083447b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:36.594290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11ca7f-d548-4f72-9cac-aedcc083447b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:36.596929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a56f79-cc90-4335-926b-ab43ffe1b0a4 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:36.652464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abdecb6-8237-4530-b397-ebe753376d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:36.653197 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abdecb6-8237-4530-b397-ebe753376d2e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:36.653486 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abdecb6-8237-4530-b397-ebe753376d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:36.653764 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abdecb6-8237-4530-b397-ebe753376d2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:36.671418 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11ca7f-d548-4f72-9cac-aedcc083447b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:36.672931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11ca7f-d548-4f72-9cac-aedcc083447b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:36.831476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abdecb6-8237-4530-b397-ebe753376d2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:36.833012 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abdecb6-8237-4530-b397-ebe753376d2e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:36.938833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 135/264] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 667 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 12:17:36.944777 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11ca7f-d548-4f72-9cac-aedcc083447b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:36.964839 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:37.015074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:37.032694 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 130/265] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:37.034038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a798c47-bdbe-4777-a940-1006b3e3c1cd None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:37.034326 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a798c47-bdbe-4777-a940-1006b3e3c1cd None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:37.034607 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a798c47-bdbe-4777-a940-1006b3e3c1cd None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:37.036217 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 131/266] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:37.037194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a798c47-bdbe-4777-a940-1006b3e3c1cd None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:37.164788 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11ca7f-d548-4f72-9cac-aedcc083447b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:37.174568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a798c47-bdbe-4777-a940-1006b3e3c1cd None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:37.211079 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9ac00-f968-489b-b9e8-7e549ae08cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:37.211724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9ac00-f968-489b-b9e8-7e549ae08cfd None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:37.212003 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9ac00-f968-489b-b9e8-7e549ae08cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:37.212279 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9ac00-f968-489b-b9e8-7e549ae08cfd None admin] PATH_INFO: `/v3/projects/34ca99f1da904d608e324f65f6571250` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:37.214218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e9ac00-f968-489b-b9e8-7e549ae08cfd None admin] RBAC: Authorizing `identity:get_project(project_id=34ca99f1da904d608e324f65f6571250)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:37.215672 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e9ac00-f968-489b-b9e8-7e549ae08cfd None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:37.217873 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 132/267] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:37 2022] GET /identity/v3/projects/34ca99f1da904d608e324f65f6571250 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:37.219002 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc0570-f5d7-4e29-a7e9-c25dceb3c5bc None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:37.219286 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc0570-f5d7-4e29-a7e9-c25dceb3c5bc None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:37.219562 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc0570-f5d7-4e29-a7e9-c25dceb3c5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:37.231005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261719c-549a-4076-9341-5cbf1aa987ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:37.231663 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261719c-549a-4076-9341-5cbf1aa987ee None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:37.231938 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261719c-549a-4076-9341-5cbf1aa987ee None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:37.232207 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261719c-549a-4076-9341-5cbf1aa987ee None admin] PATH_INFO: `/v3/projects/34ca99f1da904d608e324f65f6571250` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:37.237017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261719c-549a-4076-9341-5cbf1aa987ee None admin] RBAC: Authorizing `identity:get_project(project_id=34ca99f1da904d608e324f65f6571250)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:37.238465 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261719c-549a-4076-9341-5cbf1aa987ee None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:37.240443 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 136/268] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:36 2022] GET /identity/v3/projects/34ca99f1da904d608e324f65f6571250 => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:37.241473 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261719c-549a-4076-9341-5cbf1aa987ee None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:37.243082 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-b4cc0570-f5d7-4e29-a7e9-c25dceb3c5bc None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:37.244821 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b4cc0570-f5d7-4e29-a7e9-c25dceb3c5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:37.344355 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:37.375442 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261719c-549a-4076-9341-5cbf1aa987ee None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:37.376050 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:37.401982 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 133/269] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:37.402857 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc0570-f5d7-4e29-a7e9-c25dceb3c5bc None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:37.426921 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22cf36-a926-4c69-8a08-f1704a330a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:37.427875 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22cf36-a926-4c69-8a08-f1704a330a1f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:37.428367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22cf36-a926-4c69-8a08-f1704a330a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:37.428852 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22cf36-a926-4c69-8a08-f1704a330a1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:37.429833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22cf36-a926-4c69-8a08-f1704a330a1f None admin] RBAC: Adding query filter params (role.id=47cde553d7ee465bbe685a05c00b94e2, scope.project.id=34ca99f1da904d608e324f65f6571250, user.id=3d958849e02b4ae8bcb375e7da0949e2) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:37.430426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22cf36-a926-4c69-8a08-f1704a330a1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:37.432621 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22cf36-a926-4c69-8a08-f1704a330a1f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:37.439238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 137/270] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:17:37 2022] GET /identity/v3/role_assignments?user.id=3d958849e02b4ae8bcb375e7da0949e2&scope.project.id=34ca99f1da904d608e324f65f6571250&role.id=47cde553d7ee465bbe685a05c00b94e2 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:37.441110 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbf626-c496-4bc0-ae15-2e52a8a87e6f None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:37.441625 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbf626-c496-4bc0-ae15-2e52a8a87e6f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:37.442089 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbf626-c496-4bc0-ae15-2e52a8a87e6f None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:37.447750 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 138/271] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:37.449308 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbf626-c496-4bc0-ae15-2e52a8a87e6f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:37.571593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbf626-c496-4bc0-ae15-2e52a8a87e6f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:37.622497 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85c141-2496-4f44-9c5a-27a9cd51e0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:37.624030 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85c141-2496-4f44-9c5a-27a9cd51e0c3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:37.624540 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85c141-2496-4f44-9c5a-27a9cd51e0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:37.625017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85c141-2496-4f44-9c5a-27a9cd51e0c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:37.627043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85c141-2496-4f44-9c5a-27a9cd51e0c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:37.629320 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85c141-2496-4f44-9c5a-27a9cd51e0c3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:37.632901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 139/272] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:17:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:37.635074 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50705f-d694-4a9b-88a9-8df5b221e021 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:37.635576 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50705f-d694-4a9b-88a9-8df5b221e021 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:37.636069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50705f-d694-4a9b-88a9-8df5b221e021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:37.639702 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc0570-f5d7-4e29-a7e9-c25dceb3c5bc None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:37.653260 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:37.660900 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-af50705f-d694-4a9b-88a9-8df5b221e021 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:37.663297 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-af50705f-d694-4a9b-88a9-8df5b221e021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:37.686497 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eafd5d-3476-4348-9d23-cf682d87862d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:37.687477 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eafd5d-3476-4348-9d23-cf682d87862d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:37.687903 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eafd5d-3476-4348-9d23-cf682d87862d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:37.688369 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eafd5d-3476-4348-9d23-cf682d87862d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:37.689309 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eafd5d-3476-4348-9d23-cf682d87862d None admin] RBAC: Adding query filter params (role.id=144c8425688b409384e1e8ed8279d88e, scope.project.id=34ca99f1da904d608e324f65f6571250, user.id=3d958849e02b4ae8bcb375e7da0949e2) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:37.689783 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eafd5d-3476-4348-9d23-cf682d87862d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:37.691505 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eafd5d-3476-4348-9d23-cf682d87862d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:37.699300 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 134/273] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:17:37 2022] GET /identity/v3/role_assignments?user.id=3d958849e02b4ae8bcb375e7da0949e2&scope.project.id=34ca99f1da904d608e324f65f6571250&role.id=144c8425688b409384e1e8ed8279d88e => generated 608 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:37.700438 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eafd5d-3476-4348-9d23-cf682d87862d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:37.758051 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 140/274] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:37.761437 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f31f8-a238-4e58-8ed9-dfb5e1041c2c None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:37.762096 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f31f8-a238-4e58-8ed9-dfb5e1041c2c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:37.762571 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f31f8-a238-4e58-8ed9-dfb5e1041c2c None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:37.764348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 141/275] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:37.770901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6a9b8-bfbd-4a25-b46f-22c466c5ce6a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:37.771490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6a9b8-bfbd-4a25-b46f-22c466c5ce6a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:37.772042 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6a9b8-bfbd-4a25-b46f-22c466c5ce6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:37.791850 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:37.805351 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-19c6a9b8-bfbd-4a25-b46f-22c466c5ce6a None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:37.807040 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-19c6a9b8-bfbd-4a25-b46f-22c466c5ce6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:37.853077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eafd5d-3476-4348-9d23-cf682d87862d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:37.900931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 142/276] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:37.902897 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8cae63-2c62-40d2-80ef-6e923c1bd4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:37.903528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8cae63-2c62-40d2-80ef-6e923c1bd4e5 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:37.903807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8cae63-2c62-40d2-80ef-6e923c1bd4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:37.904080 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8cae63-2c62-40d2-80ef-6e923c1bd4e5 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:37.904792 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8cae63-2c62-40d2-80ef-6e923c1bd4e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:37.905246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6a9b8-bfbd-4a25-b46f-22c466c5ce6a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:37.906244 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8cae63-2c62-40d2-80ef-6e923c1bd4e5 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:37.937512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 135/277] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Aug 6 12:17:37 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:17:37.937512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8cae63-2c62-40d2-80ef-6e923c1bd4e5 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:38.006111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:38.050956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6a9b8-bfbd-4a25-b46f-22c466c5ce6a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:38.071292 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8cae63-2c62-40d2-80ef-6e923c1bd4e5 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:38.086863 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:38.091160 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8768f-4120-45a4-b3bb-bb4da78fc437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:38.091837 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8768f-4120-45a4-b3bb-bb4da78fc437 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:38.092138 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8768f-4120-45a4-b3bb-bb4da78fc437 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:38.092462 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8768f-4120-45a4-b3bb-bb4da78fc437 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:38.105210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8768f-4120-45a4-b3bb-bb4da78fc437 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:38.110302 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-b5b8768f-4120-45a4-b3bb-bb4da78fc437 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:38.110302 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:38.110302 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:38.110302 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:38.110302 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:38.110302 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:38.110302 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:38.110302 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:38.110302 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:38.110302 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:38.110302 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:38.110302 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:38.110302 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:38.110302 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:38.110302 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:38.110302 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:38.111354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:38.111354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:38.111354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:38.111354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:38.111354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:38.111354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:38.111354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:38.111354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:38.111354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:38.111354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:38.111354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:38.111354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:38.111354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:38.111354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:38.111354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:38.111354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:38.112247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:38.112247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:38.112247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:38.112247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:38.112247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:38.112247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:38.112247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:38.112247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:38.112247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:38.112247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:38.112247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:38.112247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:38.112247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 143/278] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:38.117246 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5654846-5d66-486f-8b74-3d24a608dc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:38.117390 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77d363-949a-4c54-af0b-a651921a6496 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:38.117714 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77d363-949a-4c54-af0b-a651921a6496 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:38.117998 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77d363-949a-4c54-af0b-a651921a6496 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:38.118198 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5654846-5d66-486f-8b74-3d24a608dc42 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:38.118491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5654846-5d66-486f-8b74-3d24a608dc42 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:38.119640 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5654846-5d66-486f-8b74-3d24a608dc42 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:38.123778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 144/279] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:38.123778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f77d363-949a-4c54-af0b-a651921a6496 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:38.135238 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5654846-5d66-486f-8b74-3d24a608dc42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:38.140943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-b5654846-5d66-486f-8b74-3d24a608dc42 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:38.140943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:38.140943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:38.140943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:38.140943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:38.140943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:38.140943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:38.140943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:38.140943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:38.140943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:38.140943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:38.140943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:38.140943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:38.140943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:38.140943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:38.140943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:38.142032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:38.142032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:38.142032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:38.142032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:38.142032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:38.142032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:38.142032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:38.142032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:38.142032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:38.142032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:38.142032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:38.142032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:38.142032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:38.142032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:38.142032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:38.142032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:38.144240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:38.144240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:38.144240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:38.144240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:38.144240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:38.144240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:38.144240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:38.144240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:38.144240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:38.144240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:38.144240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:38.144240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:17:38.144240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 136/280] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:38.144240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e88f6-9f22-425e-ae96-9a5892400cb1 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:38.145126 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e88f6-9f22-425e-ae96-9a5892400cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:38.145126 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e88f6-9f22-425e-ae96-9a5892400cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:38.164952 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-f70e88f6-9f22-425e-ae96-9a5892400cb1 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:38.167672 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f70e88f6-9f22-425e-ae96-9a5892400cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:38.255841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f77d363-949a-4c54-af0b-a651921a6496 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:38.261081 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 137/281] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:38.261937 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e88f6-9f22-425e-ae96-9a5892400cb1 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:38.285011 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:38.302819 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ddf16-a774-44d9-a0d9-3cadc465ca3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:38.303815 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ddf16-a774-44d9-a0d9-3cadc465ca3c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:38.304096 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ddf16-a774-44d9-a0d9-3cadc465ca3c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:38.304510 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ddf16-a774-44d9-a0d9-3cadc465ca3c None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:38.305387 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ddf16-a774-44d9-a0d9-3cadc465ca3c None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:38.305823 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ddf16-a774-44d9-a0d9-3cadc465ca3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:38.307510 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ddf16-a774-44d9-a0d9-3cadc465ca3c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:38.317241 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 145/282] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:38.318276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ddf16-a774-44d9-a0d9-3cadc465ca3c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:38.386659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e88f6-9f22-425e-ae96-9a5892400cb1 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:38.388457 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:38.431395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a80cb0-7379-4f5b-8b41-71356c2b8ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:38.432888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a80cb0-7379-4f5b-8b41-71356c2b8ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:38.433315 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a80cb0-7379-4f5b-8b41-71356c2b8ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:38.433881 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a80cb0-7379-4f5b-8b41-71356c2b8ad6 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:38.434846 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a80cb0-7379-4f5b-8b41-71356c2b8ad6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:38.435549 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a80cb0-7379-4f5b-8b41-71356c2b8ad6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:38.437381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a80cb0-7379-4f5b-8b41-71356c2b8ad6 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:38.446546 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 138/283] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:38.447939 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a80cb0-7379-4f5b-8b41-71356c2b8ad6 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:38.449028 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ddf16-a774-44d9-a0d9-3cadc465ca3c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:38.488265 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b58fe6-619d-4918-94ff-c2c34df36ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:38.488782 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b58fe6-619d-4918-94ff-c2c34df36ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:38.489101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b58fe6-619d-4918-94ff-c2c34df36ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:38.489377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b58fe6-619d-4918-94ff-c2c34df36ec7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:38.501407 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b58fe6-619d-4918-94ff-c2c34df36ec7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:38.508236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-58b58fe6-619d-4918-94ff-c2c34df36ec7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 12:17:38.508236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:38.508236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:38.508236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:38.508236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:38.508236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:38.508236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:38.508236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:38.508236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:38.508236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:38.508236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:38.508236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:38.508236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:38.508236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:38.508236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:38.508236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:38.509139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:38.509139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:38.509139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:38.509139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:38.509139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:38.509139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:38.509139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:38.509139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:38.509139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:38.509139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:38.509139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:38.509139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:38.509139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:38.509139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:38.509139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:38.509139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:38.511101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:38.511101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:38.511101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:38.511101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:38.511101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:38.511101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:38.511101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:38.511101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:38.511101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:38.511101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:38.511101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 12:17:38.511101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:38.511101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 146/284] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:38.511101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b58fe6-619d-4918-94ff-c2c34df36ec7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:38.573700 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:38.578637 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a80cb0-7379-4f5b-8b41-71356c2b8ad6 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:38.617887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211fe39-d380-49fa-8b96-0523d3e2d7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:38.618491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211fe39-d380-49fa-8b96-0523d3e2d7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:38.618880 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211fe39-d380-49fa-8b96-0523d3e2d7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:38.619284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211fe39-d380-49fa-8b96-0523d3e2d7b7 None admin] PATH_INFO: `/v3/users/04089b52aa2b4f0eacd85bdc4b4b370e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:38.636810 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0211fe39-d380-49fa-8b96-0523d3e2d7b7 None admin] RBAC: Authorizing `identity:get_user(user_id=04089b52aa2b4f0eacd85bdc4b4b370e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:38.638468 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0211fe39-d380-49fa-8b96-0523d3e2d7b7 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:38.641072 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 139/285] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:38 2022] GET /identity/v3/users/04089b52aa2b4f0eacd85bdc4b4b370e => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:38.641810 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211fe39-d380-49fa-8b96-0523d3e2d7b7 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:38.645978 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b58fe6-619d-4918-94ff-c2c34df36ec7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:38.671662 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:38.687164 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355d6fe-5546-4f85-a141-38310055a654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:38.687885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355d6fe-5546-4f85-a141-38310055a654 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:38.688276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355d6fe-5546-4f85-a141-38310055a654 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:38.688653 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355d6fe-5546-4f85-a141-38310055a654 None admin] PATH_INFO: `/v3/users/6359d9cca28849ce9097bd1e7a4928ab` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:38.689904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4355d6fe-5546-4f85-a141-38310055a654 None admin] RBAC: Authorizing `identity:get_user(user_id=6359d9cca28849ce9097bd1e7a4928ab)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:38.691431 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4355d6fe-5546-4f85-a141-38310055a654 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:38.693257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 147/286] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:38 2022] GET /identity/v3/users/6359d9cca28849ce9097bd1e7a4928ab => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:38.694164 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355d6fe-5546-4f85-a141-38310055a654 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:38.773092 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211fe39-d380-49fa-8b96-0523d3e2d7b7 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:38.822639 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5945b-7394-48d8-a575-3a950afba85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:38.823793 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5945b-7394-48d8-a575-3a950afba85f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:38.824558 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5945b-7394-48d8-a575-3a950afba85f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:38.825577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5945b-7394-48d8-a575-3a950afba85f None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:38.826765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c5945b-7394-48d8-a575-3a950afba85f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:38.827249 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c5945b-7394-48d8-a575-3a950afba85f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:38.829824 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c5945b-7394-48d8-a575-3a950afba85f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:38.831356 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355d6fe-5546-4f85-a141-38310055a654 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:38.841863 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 140/287] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:38.843329 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5945b-7394-48d8-a575-3a950afba85f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:38.850107 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:38.870461 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ad4e1-aff1-4b6a-9af1-3d1e28363e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:38.871218 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ad4e1-aff1-4b6a-9af1-3d1e28363e3d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:38.871572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ad4e1-aff1-4b6a-9af1-3d1e28363e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:38.871924 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ad4e1-aff1-4b6a-9af1-3d1e28363e3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:38.872845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ad4e1-aff1-4b6a-9af1-3d1e28363e3d None admin] RBAC: Adding query filter params (role.id=0fd7b75299b64c54971d9bfeaec30f4e, user.id=04089b52aa2b4f0eacd85bdc4b4b370e, scope.system=all) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:38.873221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ad4e1-aff1-4b6a-9af1-3d1e28363e3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:38.874815 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ad4e1-aff1-4b6a-9af1-3d1e28363e3d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:38.880156 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 148/288] 10.222.0.20 () {56 vars in 1308 bytes} [Sat Aug 6 12:17:38 2022] GET /identity/v3/role_assignments?user.id=04089b52aa2b4f0eacd85bdc4b4b370e&scope.system=all&role.id=0fd7b75299b64c54971d9bfeaec30f4e => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:38.881102 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ad4e1-aff1-4b6a-9af1-3d1e28363e3d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:38.947109 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:38.966968 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5945b-7394-48d8-a575-3a950afba85f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:38.995898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ad4e1-aff1-4b6a-9af1-3d1e28363e3d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:39.015584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b0959-9b6e-49e9-bb51-cb2fc471f67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:39.016652 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b0959-9b6e-49e9-bb51-cb2fc471f67b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:39.017110 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b0959-9b6e-49e9-bb51-cb2fc471f67b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:39.017557 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b0959-9b6e-49e9-bb51-cb2fc471f67b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:39.031649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed4313-70ef-4674-97a3-03dffd31e89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:39.032252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed4313-70ef-4674-97a3-03dffd31e89a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:39.032698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed4313-70ef-4674-97a3-03dffd31e89a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:39.033043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed4313-70ef-4674-97a3-03dffd31e89a None admin] PATH_INFO: `/v3/users/5c72c302441649d3a54d8272cbdc2543` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:39.034626 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed4313-70ef-4674-97a3-03dffd31e89a None admin] RBAC: Authorizing `identity:get_user(user_id=5c72c302441649d3a54d8272cbdc2543)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:39.036219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed4313-70ef-4674-97a3-03dffd31e89a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:39.043253 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 149/289] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:38 2022] GET /identity/v3/users/5c72c302441649d3a54d8272cbdc2543 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:39.044452 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed4313-70ef-4674-97a3-03dffd31e89a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:39.096437 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:39.098827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b0959-9b6e-49e9-bb51-cb2fc471f67b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:39.101053 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b0959-9b6e-49e9-bb51-cb2fc471f67b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:39.154111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed4313-70ef-4674-97a3-03dffd31e89a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:39.189771 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27c5fe-929e-4b75-ba99-3c5e6b0fe664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:39.190751 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27c5fe-929e-4b75-ba99-3c5e6b0fe664 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:39.191031 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27c5fe-929e-4b75-ba99-3c5e6b0fe664 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:39.191381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27c5fe-929e-4b75-ba99-3c5e6b0fe664 None admin] PATH_INFO: `/v3/projects/34ca99f1da904d608e324f65f6571250` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:39.192652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a27c5fe-929e-4b75-ba99-3c5e6b0fe664 None admin] RBAC: Authorizing `identity:get_project(project_id=34ca99f1da904d608e324f65f6571250)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:39.194170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a27c5fe-929e-4b75-ba99-3c5e6b0fe664 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:39.200453 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 150/290] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:39 2022] GET /identity/v3/projects/34ca99f1da904d608e324f65f6571250 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:39.201992 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:39.203165 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27c5fe-929e-4b75-ba99-3c5e6b0fe664 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:39.221887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 141/291] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:39.226939 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b0959-9b6e-49e9-bb51-cb2fc471f67b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:39.310950 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27c5fe-929e-4b75-ba99-3c5e6b0fe664 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:39.328513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:39.342384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b0959-9b6e-49e9-bb51-cb2fc471f67b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:39.348123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3152666-8296-4127-b1e9-c6d78bcf2cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:39.348961 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3152666-8296-4127-b1e9-c6d78bcf2cd6 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:39.349314 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3152666-8296-4127-b1e9-c6d78bcf2cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:39.349749 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3152666-8296-4127-b1e9-c6d78bcf2cd6 None admin] PATH_INFO: `/v3/projects/34ca99f1da904d608e324f65f6571250/users/5c72c302441649d3a54d8272cbdc2543/roles/7f46f07e73294f919cb56f7a97b324d7` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:39.351801 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3152666-8296-4127-b1e9-c6d78bcf2cd6 None admin] RBAC: Authorizing `identity:create_grant(project_id=34ca99f1da904d608e324f65f6571250, user_id=5c72c302441649d3a54d8272cbdc2543, role_id=7f46f07e73294f919cb56f7a97b324d7)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:39.353347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3152666-8296-4127-b1e9-c6d78bcf2cd6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:39.384492 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 151/292] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:17:39 2022] PUT /identity/v3/projects/34ca99f1da904d608e324f65f6571250/users/5c72c302441649d3a54d8272cbdc2543/roles/7f46f07e73294f919cb56f7a97b324d7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:17:39.386120 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442cbb90-6046-4430-9958-bad7d7ba2947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:39.386996 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cbb90-6046-4430-9958-bad7d7ba2947 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:39.387491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cbb90-6046-4430-9958-bad7d7ba2947 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:39.387977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cbb90-6046-4430-9958-bad7d7ba2947 None admin] PATH_INFO: `/v3/projects/34ca99f1da904d608e324f65f6571250` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:39.389502 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442cbb90-6046-4430-9958-bad7d7ba2947 None admin] RBAC: Authorizing `identity:get_project(project_id=34ca99f1da904d608e324f65f6571250)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:39.391291 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442cbb90-6046-4430-9958-bad7d7ba2947 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:39.393714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 142/293] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:39 2022] GET /identity/v3/projects/34ca99f1da904d608e324f65f6571250 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:39.400976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3152666-8296-4127-b1e9-c6d78bcf2cd6 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:39.538592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3152666-8296-4127-b1e9-c6d78bcf2cd6 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:39.577297 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88925f-bc40-46ff-8531-260392caa036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:39.577852 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88925f-bc40-46ff-8531-260392caa036 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:39.578133 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88925f-bc40-46ff-8531-260392caa036 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:39.578407 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88925f-bc40-46ff-8531-260392caa036 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:39.579215 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c88925f-bc40-46ff-8531-260392caa036 None admin] RBAC: Adding query filter params (role.id=0fd7b75299b64c54971d9bfeaec30f4e, scope.project.id=34ca99f1da904d608e324f65f6571250, user.id=6359d9cca28849ce9097bd1e7a4928ab) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:39.579528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c88925f-bc40-46ff-8531-260392caa036 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:39.581165 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c88925f-bc40-46ff-8531-260392caa036 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:39.588098 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 152/294] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:17:39 2022] GET /identity/v3/role_assignments?user.id=6359d9cca28849ce9097bd1e7a4928ab&scope.project.id=34ca99f1da904d608e324f65f6571250&role.id=0fd7b75299b64c54971d9bfeaec30f4e => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:40.330871 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983f4ba-ec50-4195-9ad0-3f83f481bd77 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:40.331380 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983f4ba-ec50-4195-9ad0-3f83f481bd77 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:40.331772 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983f4ba-ec50-4195-9ad0-3f83f481bd77 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:40.333540 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 143/295] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:40.339367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0f70f-79c1-4bc2-b5e2-f4fe94db464c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:40.339806 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0f70f-79c1-4bc2-b5e2-f4fe94db464c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:40.340094 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0f70f-79c1-4bc2-b5e2-f4fe94db464c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:40.358889 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-54f0f70f-79c1-4bc2-b5e2-f4fe94db464c None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:40.360139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-54f0f70f-79c1-4bc2-b5e2-f4fe94db464c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:40.396602 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:40.408240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f57e84-d200-4e5e-b64f-012c0b9c78f0 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:40.408723 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f57e84-d200-4e5e-b64f-012c0b9c78f0 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:40.408797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f57e84-d200-4e5e-b64f-012c0b9c78f0 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:40.410384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 144/296] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:17:40.415886 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8426c-f9e4-464d-9f8d-8f438983c1ab None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:40.416266 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8426c-f9e4-464d-9f8d-8f438983c1ab None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:40.416577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8426c-f9e4-464d-9f8d-8f438983c1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:40.435557 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ffc8426c-f9e4-464d-9f8d-8f438983c1ab None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:40.436404 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 153/297] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:40.436758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ffc8426c-f9e4-464d-9f8d-8f438983c1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:40.454772 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0f70f-79c1-4bc2-b5e2-f4fe94db464c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:40.490647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:40.514148 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 145/298] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:40.542286 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8426c-f9e4-464d-9f8d-8f438983c1ab None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:40.585156 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0f70f-79c1-4bc2-b5e2-f4fe94db464c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:40.636312 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ac4ce-735a-4f0c-8283-a696f2f24fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:40.637222 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ac4ce-735a-4f0c-8283-a696f2f24fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:40.637502 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ac4ce-735a-4f0c-8283-a696f2f24fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:40.637776 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ac4ce-735a-4f0c-8283-a696f2f24fb5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:40.639314 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ac4ce-735a-4f0c-8283-a696f2f24fb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:40.641347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ac4ce-735a-4f0c-8283-a696f2f24fb5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:40.644379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 154/299] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:17:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:40.649969 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ac4ce-735a-4f0c-8283-a696f2f24fb5 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:40.659857 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8426c-f9e4-464d-9f8d-8f438983c1ab None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:40.695414 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b6e96-fd89-4f9a-8b1f-dd6fbdde06a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:40.696095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b6e96-fd89-4f9a-8b1f-dd6fbdde06a5 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:40.696413 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b6e96-fd89-4f9a-8b1f-dd6fbdde06a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:40.696684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b6e96-fd89-4f9a-8b1f-dd6fbdde06a5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:40.706959 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:40.712725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160b6e96-fd89-4f9a-8b1f-dd6fbdde06a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:40.720629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-160b6e96-fd89-4f9a-8b1f-dd6fbdde06a5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 12:17:40.720629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:40.720629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:40.720629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:40.720629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:40.720629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:40.720629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:40.720629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:40.720629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:40.720629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:40.720629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:40.720629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:40.720629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:40.720629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:40.720629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:40.720629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:40.721516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:40.721516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:40.721516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:40.721516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:40.721516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:40.721516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:40.721516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:40.721516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:40.721516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:40.721516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:40.721516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:40.721516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:40.721516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:40.721516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:40.721516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:40.721516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:40.722335 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:40.722335 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:40.722335 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:40.722335 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:40.722335 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:40.722335 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:40.722335 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:40.722335 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:40.722335 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:40.722335 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:40.722335 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 12:17:40.722335 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:17:40.725674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 146/300] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:17:40.732266 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b6e96-fd89-4f9a-8b1f-dd6fbdde06a5 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:40.778559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:40.790134 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ac4ce-735a-4f0c-8283-a696f2f24fb5 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:40.828921 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb3cd8-d802-4bd3-802e-dd4811a65a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:40.829656 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb3cd8-d802-4bd3-802e-dd4811a65a7b None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:40.830003 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb3cd8-d802-4bd3-802e-dd4811a65a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:40.830467 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb3cd8-d802-4bd3-802e-dd4811a65a7b None admin] PATH_INFO: `/v3/groups` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:40.831234 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb3cd8-d802-4bd3-802e-dd4811a65a7b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:40.832948 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb3cd8-d802-4bd3-802e-dd4811a65a7b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:40.840851 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b6e96-fd89-4f9a-8b1f-dd6fbdde06a5 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:40.854680 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 155/301] 10.222.0.20 () {60 vars in 1134 bytes} [Sat Aug 6 12:17:40 2022] POST /identity/v3/groups => generated 229 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:17:40.895890 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b659170-77ae-4c93-bbbf-e51c0d5fd959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:40.896931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b659170-77ae-4c93-bbbf-e51c0d5fd959 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:40.897481 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b659170-77ae-4c93-bbbf-e51c0d5fd959 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:40.897948 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b659170-77ae-4c93-bbbf-e51c0d5fd959 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:40.898996 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b659170-77ae-4c93-bbbf-e51c0d5fd959 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:40.899555 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b659170-77ae-4c93-bbbf-e51c0d5fd959 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:40.901744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b659170-77ae-4c93-bbbf-e51c0d5fd959 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:40.911933 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 147/302] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:40.918858 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb3cd8-d802-4bd3-802e-dd4811a65a7b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:41.032763 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:41.037612 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb3cd8-d802-4bd3-802e-dd4811a65a7b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:41.075256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696f730-a62b-46f0-8065-ba270954748b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:41.075894 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696f730-a62b-46f0-8065-ba270954748b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:41.076171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696f730-a62b-46f0-8065-ba270954748b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:41.076474 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696f730-a62b-46f0-8065-ba270954748b None admin] PATH_INFO: `/v3/users/e78afc5b30424635bc2e1f0a893c14ae` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:41.089805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696f730-a62b-46f0-8065-ba270954748b None admin] RBAC: Authorizing `identity:get_user(user_id=e78afc5b30424635bc2e1f0a893c14ae)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:41.091307 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696f730-a62b-46f0-8065-ba270954748b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:41.093209 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 156/303] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:40 2022] GET /identity/v3/users/e78afc5b30424635bc2e1f0a893c14ae => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:41.098958 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b659170-77ae-4c93-bbbf-e51c0d5fd959 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:41.213831 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b659170-77ae-4c93-bbbf-e51c0d5fd959 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:41.216630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:41.251681 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970259b-8733-43e5-8277-9e3ce3f2de8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:41.252746 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970259b-8733-43e5-8277-9e3ce3f2de8e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:41.253252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970259b-8733-43e5-8277-9e3ce3f2de8e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:41.253720 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970259b-8733-43e5-8277-9e3ce3f2de8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:41.254701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8970259b-8733-43e5-8277-9e3ce3f2de8e None admin] RBAC: Adding query filter params (role.id=7f46f07e73294f919cb56f7a97b324d7, user.id=e78afc5b30424635bc2e1f0a893c14ae, scope.system=all) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:41.255289 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8970259b-8733-43e5-8277-9e3ce3f2de8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:41.257173 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603d151-8baf-40c4-9fe5-bb3e478bca02 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:41.257462 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603d151-8baf-40c4-9fe5-bb3e478bca02 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:41.257741 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603d151-8baf-40c4-9fe5-bb3e478bca02 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:41.257919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8970259b-8733-43e5-8277-9e3ce3f2de8e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:41.259246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 157/304] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:17:41.264912 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc3349-6d84-4c11-8728-19a8ce7c1332 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:41.265156 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc3349-6d84-4c11-8728-19a8ce7c1332 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:41.265534 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc3349-6d84-4c11-8728-19a8ce7c1332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:41.265766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 148/305] 10.222.0.20 () {56 vars in 1308 bytes} [Sat Aug 6 12:17:41 2022] GET /identity/v3/role_assignments?user.id=e78afc5b30424635bc2e1f0a893c14ae&scope.system=all&role.id=7f46f07e73294f919cb56f7a97b324d7 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:41.291700 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b7cc3349-6d84-4c11-8728-19a8ce7c1332 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:41.292984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b7cc3349-6d84-4c11-8728-19a8ce7c1332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:41.373961 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 158/306] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 12:17:41.401690 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970259b-8733-43e5-8277-9e3ce3f2de8e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:41.517280 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970259b-8733-43e5-8277-9e3ce3f2de8e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:41.557802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e2d23-815a-4075-8f54-f3a6cbf18d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:41.558435 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e2d23-815a-4075-8f54-f3a6cbf18d94 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:41.558711 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e2d23-815a-4075-8f54-f3a6cbf18d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:41.559041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e2d23-815a-4075-8f54-f3a6cbf18d94 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:41.569789 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e2d23-815a-4075-8f54-f3a6cbf18d94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:41.575218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-721e2d23-815a-4075-8f54-f3a6cbf18d94 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:41.575218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:41.575218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:41.575218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:41.575218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:41.575218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:41.575218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:41.575218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:41.575218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:41.575218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:41.575218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:41.575218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:41.575218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:41.575218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:41.575218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:41.575218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:41.576063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:41.576063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:41.576063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:41.576063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:41.576063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:41.576063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:41.576063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:41.576063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:41.576063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:41.576063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:41.576063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:41.576063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:41.576063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:41.576063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:41.576063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:41.576063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:41.577068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:41.577068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:41.577068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:41.577068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:41.577068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:41.577068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:41.577068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:41.577068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:41.577068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:41.577068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:41.577068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:41.577068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:17:41.577068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 149/307] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:41.582207 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc3349-6d84-4c11-8728-19a8ce7c1332 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:41.670304 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:41.686027 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc3349-6d84-4c11-8728-19a8ce7c1332 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:41.721961 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bdf9da-4bb0-4692-9759-89d000094139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:41.722591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdf9da-4bb0-4692-9759-89d000094139 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:41.722874 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdf9da-4bb0-4692-9759-89d000094139 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:41.723149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdf9da-4bb0-4692-9759-89d000094139 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:41.723864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bdf9da-4bb0-4692-9759-89d000094139 None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:41.724165 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bdf9da-4bb0-4692-9759-89d000094139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:41.725572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bdf9da-4bb0-4692-9759-89d000094139 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:41.734795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 159/308] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:41.740350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e2d23-815a-4075-8f54-f3a6cbf18d94 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:41.758991 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198ea50-710c-4823-bcb5-2c73d37d3ee1 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:41.759335 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198ea50-710c-4823-bcb5-2c73d37d3ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:41.759618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198ea50-710c-4823-bcb5-2c73d37d3ee1 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:41.761441 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 160/309] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:17:41.766184 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206a132-bf06-44dd-a5fe-710dc0e4faf3 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:41.766467 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206a132-bf06-44dd-a5fe-710dc0e4faf3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:41.766739 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206a132-bf06-44dd-a5fe-710dc0e4faf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:41.780083 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:41.786014 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0206a132-bf06-44dd-a5fe-710dc0e4faf3 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:41.787205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0206a132-bf06-44dd-a5fe-710dc0e4faf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:41.864599 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e2d23-815a-4075-8f54-f3a6cbf18d94 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:41.878640 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 161/310] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 12:17:41.901942 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206a132-bf06-44dd-a5fe-710dc0e4faf3 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:41.904765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3185a6a-e2c2-4734-8257-6b2c14368d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:41.905482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3185a6a-e2c2-4734-8257-6b2c14368d2a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:41.905833 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3185a6a-e2c2-4734-8257-6b2c14368d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:41.906200 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3185a6a-e2c2-4734-8257-6b2c14368d2a None admin] PATH_INFO: `/v3/users/04089b52aa2b4f0eacd85bdc4b4b370e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:41.907498 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3185a6a-e2c2-4734-8257-6b2c14368d2a None admin] RBAC: Authorizing `identity:get_user(user_id=04089b52aa2b4f0eacd85bdc4b4b370e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:41.909135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3185a6a-e2c2-4734-8257-6b2c14368d2a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:41.910901 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 150/311] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:41 2022] GET /identity/v3/users/04089b52aa2b4f0eacd85bdc4b4b370e => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:41.916823 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3185a6a-e2c2-4734-8257-6b2c14368d2a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:42.005677 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:42.019599 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:42.026736 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3185a6a-e2c2-4734-8257-6b2c14368d2a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:42.037648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206a132-bf06-44dd-a5fe-710dc0e4faf3 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:42.068054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e6da6-0a72-47b2-8d14-a4c3fa7d4522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:42.069106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e6da6-0a72-47b2-8d14-a4c3fa7d4522 None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:42.069458 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e6da6-0a72-47b2-8d14-a4c3fa7d4522 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:42.070234 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e6da6-0a72-47b2-8d14-a4c3fa7d4522 None admin] PATH_INFO: `/v3/system/users/04089b52aa2b4f0eacd85bdc4b4b370e/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:42.072041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e6da6-0a72-47b2-8d14-a4c3fa7d4522 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=04089b52aa2b4f0eacd85bdc4b4b370e, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:42.075054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e6da6-0a72-47b2-8d14-a4c3fa7d4522 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:42.082905 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae465f47-2d82-4d13-ac20-86a0e82f17b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:42.083610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae465f47-2d82-4d13-ac20-86a0e82f17b8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:42.083940 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae465f47-2d82-4d13-ac20-86a0e82f17b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:42.084186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae465f47-2d82-4d13-ac20-86a0e82f17b8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:42.096559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 151/312] 10.222.0.20 () {58 vars in 1334 bytes} [Sat Aug 6 12:17:41 2022] PUT /identity/v3/system/users/04089b52aa2b4f0eacd85bdc4b4b370e/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:17:42.104651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae465f47-2d82-4d13-ac20-86a0e82f17b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:42.110398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-ae465f47-2d82-4d13-ac20-86a0e82f17b8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 12:17:42.110398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:42.110398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:42.110398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:42.110398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:42.110398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:42.110398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:42.110398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:42.110398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:42.110398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:42.110398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:42.110398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:42.110398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:42.110398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:42.110398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:42.110398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:42.111357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:42.111357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:42.111357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:42.111357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:42.111357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:42.111357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:42.111357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:42.111357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:42.111357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:42.111357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:42.111357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:42.111357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:42.111357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:42.111357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:42.111357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:42.111357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:42.112149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:42.112149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:42.112149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:42.112149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:42.112149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:42.112149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:42.112149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:42.112149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:42.112149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:42.112149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:42.112149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 12:17:42.112149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:42.112149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 162/313] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:17:42.119458 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e6da6-0a72-47b2-8d14-a4c3fa7d4522 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:42.238239 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e6da6-0a72-47b2-8d14-a4c3fa7d4522 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:42.277334 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a675b-5e99-47d9-9ece-70ec501fa9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:42.277878 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a675b-5e99-47d9-9ece-70ec501fa9da None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:42.278153 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a675b-5e99-47d9-9ece-70ec501fa9da None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:42.278424 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a675b-5e99-47d9-9ece-70ec501fa9da None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:42.279131 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a675b-5e99-47d9-9ece-70ec501fa9da None admin] RBAC: Adding query filter params (name=reader) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:42.279428 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a675b-5e99-47d9-9ece-70ec501fa9da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:42.280832 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a675b-5e99-47d9-9ece-70ec501fa9da None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:42.289427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 152/314] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:42.295393 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae465f47-2d82-4d13-ac20-86a0e82f17b8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:42.398376 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae465f47-2d82-4d13-ac20-86a0e82f17b8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:42.435197 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02051a-a6e4-43e5-a406-b2abfa895bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:42.435834 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02051a-a6e4-43e5-a406-b2abfa895bbd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:42.436115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02051a-a6e4-43e5-a406-b2abfa895bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:42.436409 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02051a-a6e4-43e5-a406-b2abfa895bbd None admin] PATH_INFO: `/v3/users/5c72c302441649d3a54d8272cbdc2543` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:42.437625 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee02051a-a6e4-43e5-a406-b2abfa895bbd None admin] RBAC: Authorizing `identity:get_user(user_id=5c72c302441649d3a54d8272cbdc2543)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:42.439121 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee02051a-a6e4-43e5-a406-b2abfa895bbd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:42.440927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 163/315] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:42 2022] GET /identity/v3/users/5c72c302441649d3a54d8272cbdc2543 => generated 297 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:42.446353 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a675b-5e99-47d9-9ece-70ec501fa9da None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:42.477008 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:42.560267 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a675b-5e99-47d9-9ece-70ec501fa9da None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:42.595408 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0fa07-0c75-4995-9a98-31feec58627f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:42.596034 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0fa07-0c75-4995-9a98-31feec58627f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:42.596345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0fa07-0c75-4995-9a98-31feec58627f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:42.596667 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0fa07-0c75-4995-9a98-31feec58627f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:42.667974 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0fa07-0c75-4995-9a98-31feec58627f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:42.669851 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0fa07-0c75-4995-9a98-31feec58627f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:42.692850 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:42.774257 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 153/316] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:42.779980 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02051a-a6e4-43e5-a406-b2abfa895bbd None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:42.789490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:42.889643 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02051a-a6e4-43e5-a406-b2abfa895bbd None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:42.925301 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413c0f5-9fb3-43f0-abf0-3309d80c8867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:42.926348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413c0f5-9fb3-43f0-abf0-3309d80c8867 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:42.926837 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413c0f5-9fb3-43f0-abf0-3309d80c8867 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:42.927309 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413c0f5-9fb3-43f0-abf0-3309d80c8867 None admin] PATH_INFO: `/v3/projects/34ca99f1da904d608e324f65f6571250` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:42.928699 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413c0f5-9fb3-43f0-abf0-3309d80c8867 None admin] RBAC: Authorizing `identity:get_project(project_id=34ca99f1da904d608e324f65f6571250)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:42.930491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413c0f5-9fb3-43f0-abf0-3309d80c8867 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:42.932989 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 164/317] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:42 2022] GET /identity/v3/projects/34ca99f1da904d608e324f65f6571250 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:42.938887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0fa07-0c75-4995-9a98-31feec58627f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:43.042012 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0fa07-0c75-4995-9a98-31feec58627f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:43.065890 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:43.077431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7e6f1-6be3-44a9-8a38-c9f31c5fce8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:43.078273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7e6f1-6be3-44a9-8a38-c9f31c5fce8c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:43.078550 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7e6f1-6be3-44a9-8a38-c9f31c5fce8c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:43.078823 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7e6f1-6be3-44a9-8a38-c9f31c5fce8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:43.079611 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7e6f1-6be3-44a9-8a38-c9f31c5fce8c None admin] RBAC: Adding query filter params (role.id=7f46f07e73294f919cb56f7a97b324d7, scope.project.id=34ca99f1da904d608e324f65f6571250, user.id=5c72c302441649d3a54d8272cbdc2543) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:43.079918 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7e6f1-6be3-44a9-8a38-c9f31c5fce8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:43.081379 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7e6f1-6be3-44a9-8a38-c9f31c5fce8c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:43.087673 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 154/318] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:17:42 2022] GET /identity/v3/role_assignments?user.id=5c72c302441649d3a54d8272cbdc2543&scope.project.id=34ca99f1da904d608e324f65f6571250&role.id=7f46f07e73294f919cb56f7a97b324d7 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:43.099159 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ddb0a-33b5-45a7-b1dd-2b9f74243d27 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:43.099468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ddb0a-33b5-45a7-b1dd-2b9f74243d27 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:43.099718 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ddb0a-33b5-45a7-b1dd-2b9f74243d27 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:43.101173 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 165/319] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:43.107613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce758b1-f468-47c0-be1a-cdd7cb650539 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:43.107899 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce758b1-f468-47c0-be1a-cdd7cb650539 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:43.108175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce758b1-f468-47c0-be1a-cdd7cb650539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:43.129539 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-bce758b1-f468-47c0-be1a-cdd7cb650539 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:43.131195 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-bce758b1-f468-47c0-be1a-cdd7cb650539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:43.206483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 155/320] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:43.230704 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ddb0a-33b5-45a7-b1dd-2b9f74243d27 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:43.318541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:43.349245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06ddb0a-33b5-45a7-b1dd-2b9f74243d27 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:43.371970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f406464-401d-4d1a-8a6c-39c8e96e840c None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:43.372603 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f406464-401d-4d1a-8a6c-39c8e96e840c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:43.373080 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f406464-401d-4d1a-8a6c-39c8e96e840c None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:43.374915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 156/321] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:43.379643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165505c-fcf4-4d57-81fd-ee3d33a80ffe None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:43.380221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165505c-fcf4-4d57-81fd-ee3d33a80ffe None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:43.382819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165505c-fcf4-4d57-81fd-ee3d33a80ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:43.397038 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b4620-dcc4-4f50-b28e-6137b34da85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:43.397697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b4620-dcc4-4f50-b28e-6137b34da85d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:43.397976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b4620-dcc4-4f50-b28e-6137b34da85d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:43.398248 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b4620-dcc4-4f50-b28e-6137b34da85d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:43.399424 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b4620-dcc4-4f50-b28e-6137b34da85d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:43.401496 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b4620-dcc4-4f50-b28e-6137b34da85d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:43.403253 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 166/322] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:17:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:43.405394 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-d165505c-fcf4-4d57-81fd-ee3d33a80ffe None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:43.406576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d165505c-fcf4-4d57-81fd-ee3d33a80ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:43.408595 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b4620-dcc4-4f50-b28e-6137b34da85d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:43.476448 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 157/323] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 12:17:43.499282 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165505c-fcf4-4d57-81fd-ee3d33a80ffe None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:43.522533 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b4620-dcc4-4f50-b28e-6137b34da85d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:43.545849 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:43.562664 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271db637-8d06-4bae-b62f-cc66fc8c46f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:43.563282 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271db637-8d06-4bae-b62f-cc66fc8c46f6 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:43.563566 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271db637-8d06-4bae-b62f-cc66fc8c46f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:43.563991 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271db637-8d06-4bae-b62f-cc66fc8c46f6 None admin] PATH_INFO: `/v3/groups` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:43.564596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271db637-8d06-4bae-b62f-cc66fc8c46f6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:43.565934 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271db637-8d06-4bae-b62f-cc66fc8c46f6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:43.582761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 167/324] 10.222.0.20 () {60 vars in 1134 bytes} [Sat Aug 6 12:17:43 2022] POST /identity/v3/groups => generated 226 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 12:17:43.610444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165505c-fcf4-4d57-81fd-ee3d33a80ffe None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:43.648851 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c9294-92f1-4e49-b4c0-b3b0b9054aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:43.649511 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c9294-92f1-4e49-b4c0-b3b0b9054aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:43.649801 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c9294-92f1-4e49-b4c0-b3b0b9054aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:43.650082 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c9294-92f1-4e49-b4c0-b3b0b9054aa0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:43.662213 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c9294-92f1-4e49-b4c0-b3b0b9054aa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:43.667455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-0d2c9294-92f1-4e49-b4c0-b3b0b9054aa0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 12:17:43.667455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:43.667455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:43.667455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:43.667455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:43.667455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:43.667455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:43.667455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:43.667455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:43.667455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:43.667455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:43.667455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:43.667455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:43.667455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:43.667455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:43.667455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:43.668714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:43.668714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:43.668714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:43.668714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:43.668714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:43.668714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:43.668714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:43.668714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:43.668714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:43.668714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:43.668714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:43.668714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:43.668714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:43.668714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:43.668714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:43.668714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:43.669659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:43.669659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:43.669659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:43.669659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:43.669659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:43.669659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:43.669659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:43.669659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:43.669659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:43.669659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:43.669659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 12:17:43.669659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:17:43.669659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 158/325] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:17:43.675048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271db637-8d06-4bae-b62f-cc66fc8c46f6 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:43.678344 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:43.792585 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271db637-8d06-4bae-b62f-cc66fc8c46f6 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:43.828942 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed747c-11b5-4f0f-8731-69315c929fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:43.829568 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed747c-11b5-4f0f-8731-69315c929fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:43.829848 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed747c-11b5-4f0f-8731-69315c929fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:43.830119 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed747c-11b5-4f0f-8731-69315c929fc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:43.830838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed747c-11b5-4f0f-8731-69315c929fc0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:43.831199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed747c-11b5-4f0f-8731-69315c929fc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:43.832599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed747c-11b5-4f0f-8731-69315c929fc0 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:43.847992 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 168/326] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:43.849385 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c9294-92f1-4e49-b4c0-b3b0b9054aa0 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:43.957500 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c9294-92f1-4e49-b4c0-b3b0b9054aa0 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:43.966419 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:44.004814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e97ca7-40bb-4dfd-b0db-920aad6ddd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:44.005465 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e97ca7-40bb-4dfd-b0db-920aad6ddd58 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:44.005745 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e97ca7-40bb-4dfd-b0db-920aad6ddd58 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:44.006019 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e97ca7-40bb-4dfd-b0db-920aad6ddd58 None admin] PATH_INFO: `/v3/users/e78afc5b30424635bc2e1f0a893c14ae` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:44.007203 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e97ca7-40bb-4dfd-b0db-920aad6ddd58 None admin] RBAC: Authorizing `identity:get_user(user_id=e78afc5b30424635bc2e1f0a893c14ae)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:44.008698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e97ca7-40bb-4dfd-b0db-920aad6ddd58 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:44.010656 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 159/327] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:43 2022] GET /identity/v3/users/e78afc5b30424635bc2e1f0a893c14ae => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:44.016124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed747c-11b5-4f0f-8731-69315c929fc0 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:44.091966 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:44.134565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed747c-11b5-4f0f-8731-69315c929fc0 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:44.172882 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d578b11-78a4-4a5b-8e0e-81ab92b1e983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:44.173549 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d578b11-78a4-4a5b-8e0e-81ab92b1e983 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:44.173845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d578b11-78a4-4a5b-8e0e-81ab92b1e983 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:44.174163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d578b11-78a4-4a5b-8e0e-81ab92b1e983 None admin] PATH_INFO: `/v3/system/users/e78afc5b30424635bc2e1f0a893c14ae/roles/7f46f07e73294f919cb56f7a97b324d7` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:44.175649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d578b11-78a4-4a5b-8e0e-81ab92b1e983 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e78afc5b30424635bc2e1f0a893c14ae, role_id=7f46f07e73294f919cb56f7a97b324d7)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:44.177056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d578b11-78a4-4a5b-8e0e-81ab92b1e983 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:44.189937 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 169/328] 10.222.0.20 () {58 vars in 1334 bytes} [Sat Aug 6 12:17:44 2022] PUT /identity/v3/system/users/e78afc5b30424635bc2e1f0a893c14ae/roles/7f46f07e73294f919cb56f7a97b324d7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:17:44.274021 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f5af2-7198-4fa9-87bc-52c28cb6139e None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:44.274304 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f5af2-7198-4fa9-87bc-52c28cb6139e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:44.274581 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f5af2-7198-4fa9-87bc-52c28cb6139e None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:44.276248 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 160/329] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:44.281939 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70bd69-308b-4bda-a8e2-fc6c6afd6907 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:44.282226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70bd69-308b-4bda-a8e2-fc6c6afd6907 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:44.282531 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70bd69-308b-4bda-a8e2-fc6c6afd6907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:44.305306 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9b70bd69-308b-4bda-a8e2-fc6c6afd6907 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:44.306625 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9b70bd69-308b-4bda-a8e2-fc6c6afd6907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:44.394852 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 170/330] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:44.418371 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f5af2-7198-4fa9-87bc-52c28cb6139e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:44.534793 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f5af2-7198-4fa9-87bc-52c28cb6139e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:44.574854 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c430de-b7fe-4fc2-b8c8-cb8503a6b8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:44.575499 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c430de-b7fe-4fc2-b8c8-cb8503a6b8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:44.575783 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c430de-b7fe-4fc2-b8c8-cb8503a6b8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:44.576055 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c430de-b7fe-4fc2-b8c8-cb8503a6b8f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:44.590968 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c430de-b7fe-4fc2-b8c8-cb8503a6b8f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:44.597668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-d2c430de-b7fe-4fc2-b8c8-cb8503a6b8f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:44.597668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:44.597668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:44.597668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:44.597668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:44.597668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:44.597668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:44.597668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:44.597668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:44.597668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:44.597668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:44.597668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:44.597668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:44.597668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:44.597668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:44.597668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:44.598516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:44.598516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:44.598516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:44.598516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:44.598516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:44.598516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:44.598516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:44.598516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:44.598516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:44.598516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:44.598516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:44.598516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:44.598516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:44.598516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:44.598516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:44.598516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:44.599297 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:44.599297 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:44.599297 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:44.599297 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:44.599297 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:44.599297 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:44.599297 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:44.599297 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:44.599297 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:44.599297 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:44.599297 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:44.599297 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:17:44.599297 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 161/331] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:44.605881 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70bd69-308b-4bda-a8e2-fc6c6afd6907 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:44.685413 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:44.737907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70bd69-308b-4bda-a8e2-fc6c6afd6907 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:44.786542 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e2b35-7975-475d-ae8b-540cad8be935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:44.787179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e2b35-7975-475d-ae8b-540cad8be935 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:44.787522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e2b35-7975-475d-ae8b-540cad8be935 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:44.787804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e2b35-7975-475d-ae8b-540cad8be935 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:44.788663 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e2b35-7975-475d-ae8b-540cad8be935 None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:44.788973 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e2b35-7975-475d-ae8b-540cad8be935 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:44.790369 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e2b35-7975-475d-ae8b-540cad8be935 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:44.800406 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 171/332] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:44.806238 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c430de-b7fe-4fc2-b8c8-cb8503a6b8f0 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:44.857997 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:44.939749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c430de-b7fe-4fc2-b8c8-cb8503a6b8f0 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:44.979144 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79f46a-5d04-4dff-8143-1032dd3b77c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:44.979795 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79f46a-5d04-4dff-8143-1032dd3b77c9 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:44.980078 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79f46a-5d04-4dff-8143-1032dd3b77c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:44.980378 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79f46a-5d04-4dff-8143-1032dd3b77c9 None admin] PATH_INFO: `/v3/users/04089b52aa2b4f0eacd85bdc4b4b370e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:44.981583 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79f46a-5d04-4dff-8143-1032dd3b77c9 None admin] RBAC: Authorizing `identity:get_user(user_id=04089b52aa2b4f0eacd85bdc4b4b370e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:44.983095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79f46a-5d04-4dff-8143-1032dd3b77c9 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:44.985048 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 162/333] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:44 2022] GET /identity/v3/users/04089b52aa2b4f0eacd85bdc4b4b370e => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:44.991652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e2b35-7975-475d-ae8b-540cad8be935 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:45.152471 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e2b35-7975-475d-ae8b-540cad8be935 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:45.179864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:45.195997 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22676ff-369e-4a4d-a28d-1fd02b45a5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:45.196702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22676ff-369e-4a4d-a28d-1fd02b45a5b3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:45.196992 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22676ff-369e-4a4d-a28d-1fd02b45a5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:45.197801 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22676ff-369e-4a4d-a28d-1fd02b45a5b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:45.198597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22676ff-369e-4a4d-a28d-1fd02b45a5b3 None admin] RBAC: Adding query filter params (role.id=0fd7b75299b64c54971d9bfeaec30f4e, user.id=04089b52aa2b4f0eacd85bdc4b4b370e, scope.system=all) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:45.198911 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22676ff-369e-4a4d-a28d-1fd02b45a5b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:45.200460 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22676ff-369e-4a4d-a28d-1fd02b45a5b3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:45.215678 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 172/334] 10.222.0.20 () {56 vars in 1308 bytes} [Sat Aug 6 12:17:44 2022] GET /identity/v3/role_assignments?user.id=04089b52aa2b4f0eacd85bdc4b4b370e&scope.system=all&role.id=0fd7b75299b64c54971d9bfeaec30f4e => generated 510 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:45.945714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a62cc-6eaf-408e-97b8-944076367e1b None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:45.946748 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a62cc-6eaf-408e-97b8-944076367e1b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:45.947763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a62cc-6eaf-408e-97b8-944076367e1b None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:45.949652 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 163/335] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:45.956486 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb59cda-dc22-4d32-9c70-5e4212eb49c9 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:45.957335 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb59cda-dc22-4d32-9c70-5e4212eb49c9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:45.957833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb59cda-dc22-4d32-9c70-5e4212eb49c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:45.980373 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-cbb59cda-dc22-4d32-9c70-5e4212eb49c9 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:45.981894 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cbb59cda-dc22-4d32-9c70-5e4212eb49c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:46.075176 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 173/336] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 12:17:46.111119 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a62cc-6eaf-408e-97b8-944076367e1b None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:46.243367 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a62cc-6eaf-408e-97b8-944076367e1b None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:46.245868 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:46.286596 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398bfaa9-4d0b-46d2-ba98-b5c5b6c0589a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:46.287611 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398bfaa9-4d0b-46d2-ba98-b5c5b6c0589a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:46.288112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398bfaa9-4d0b-46d2-ba98-b5c5b6c0589a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:46.288619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398bfaa9-4d0b-46d2-ba98-b5c5b6c0589a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:46.302984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398bfaa9-4d0b-46d2-ba98-b5c5b6c0589a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:46.308907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-398bfaa9-4d0b-46d2-ba98-b5c5b6c0589a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:46.308907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:46.308907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:46.308907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:46.308907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:46.308907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:46.308907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:46.308907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:46.308907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:46.308907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:46.308907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:46.308907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:46.308907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:46.308907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:46.308907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:46.308907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:46.309796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:46.309796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:46.309796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:46.309796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:46.309796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:46.309796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:46.309796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:46.309796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:46.309796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:46.309796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:46.309796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:46.309796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:46.309796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:46.309796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:46.309796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:46.309796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:46.310593 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:46.310593 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:46.310593 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:46.310593 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:46.310593 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:46.310593 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:46.310593 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:46.310593 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:46.310593 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:46.310593 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:46.310593 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:46.310593 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:17:46.313220 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 164/337] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:46.319440 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb59cda-dc22-4d32-9c70-5e4212eb49c9 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:46.446786 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb59cda-dc22-4d32-9c70-5e4212eb49c9 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:46.459156 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:46.492048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b1a88-f54c-4c2d-8f49-1035ae72ab4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:46.493046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b1a88-f54c-4c2d-8f49-1035ae72ab4e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:46.493603 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b1a88-f54c-4c2d-8f49-1035ae72ab4e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:46.494598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b1a88-f54c-4c2d-8f49-1035ae72ab4e None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:46.495525 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b1a88-f54c-4c2d-8f49-1035ae72ab4e None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:46.496030 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b1a88-f54c-4c2d-8f49-1035ae72ab4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:46.497628 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b1a88-f54c-4c2d-8f49-1035ae72ab4e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:46.506847 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 174/338] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:46.512783 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398bfaa9-4d0b-46d2-ba98-b5c5b6c0589a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:46.653525 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398bfaa9-4d0b-46d2-ba98-b5c5b6c0589a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:46.695033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab0ff1-7551-45bf-a564-edea7bbca852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:46.695704 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab0ff1-7551-45bf-a564-edea7bbca852 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:46.696001 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab0ff1-7551-45bf-a564-edea7bbca852 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:46.696272 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab0ff1-7551-45bf-a564-edea7bbca852 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:46.735983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:46.782577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab0ff1-7551-45bf-a564-edea7bbca852 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:46.784012 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab0ff1-7551-45bf-a564-edea7bbca852 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:46.838208 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedfaec-de87-4e88-b14e-bf77afdf8535 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:46.838840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedfaec-de87-4e88-b14e-bf77afdf8535 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:46.839316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedfaec-de87-4e88-b14e-bf77afdf8535 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:46.840907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 175/339] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:46.846108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70a001-940b-4d91-821e-2dd6294c1ff6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:46.846635 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70a001-940b-4d91-821e-2dd6294c1ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:46.847133 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70a001-940b-4d91-821e-2dd6294c1ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:46.879665 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2b70a001-940b-4d91-821e-2dd6294c1ff6 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:46.881418 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2b70a001-940b-4d91-821e-2dd6294c1ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:47.001519 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 165/340] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:47.033936 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab0ff1-7551-45bf-a564-edea7bbca852 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:47.048755 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 176/341] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 12:17:47.086290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70a001-940b-4d91-821e-2dd6294c1ff6 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:47.160668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:47.253796 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:47.260385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70a001-940b-4d91-821e-2dd6294c1ff6 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:47.264498 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab0ff1-7551-45bf-a564-edea7bbca852 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:47.336983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2f19c-738e-4dce-9e45-11d94b2c7bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:47.337552 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2f19c-738e-4dce-9e45-11d94b2c7bbc None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:47.337874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2f19c-738e-4dce-9e45-11d94b2c7bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:47.338155 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2f19c-738e-4dce-9e45-11d94b2c7bbc None admin] PATH_INFO: `/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:47.339336 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2f19c-738e-4dce-9e45-11d94b2c7bbc None admin] RBAC: Authorizing `identity:get_project(project_id=5a84e3fab8334ecca0cd4aae83cb5317)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:47.340858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2f19c-738e-4dce-9e45-11d94b2c7bbc None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:47.343525 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 166/342] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:47 2022] GET /identity/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317 => generated 298 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:47.349201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2f19c-738e-4dce-9e45-11d94b2c7bbc None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:47.358629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3145bc2-4362-4c67-bbe2-79b06440effe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:47.359285 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3145bc2-4362-4c67-bbe2-79b06440effe None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:47.359565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3145bc2-4362-4c67-bbe2-79b06440effe None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:47.359841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3145bc2-4362-4c67-bbe2-79b06440effe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:47.385619 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3145bc2-4362-4c67-bbe2-79b06440effe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:47.390825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-e3145bc2-4362-4c67-bbe2-79b06440effe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 12:17:47.390825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:47.390825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:47.390825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:47.390825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:47.390825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:47.390825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:47.390825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:47.390825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:47.390825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:47.390825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:47.390825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:47.390825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:47.390825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:47.390825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:47.390825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:47.391683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:47.391683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:47.391683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:47.391683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:47.391683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:47.391683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:47.391683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:47.391683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:47.391683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:47.391683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:47.391683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:47.391683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:47.391683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:47.391683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:47.391683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:47.391683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:47.392513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:47.392513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:47.392513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:47.392513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:47.392513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:47.392513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:47.392513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:47.392513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:47.392513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:47.392513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:47.392513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 06 12:17:47.392513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:47.392513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 177/343] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:17:47.399955 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3145bc2-4362-4c67-bbe2-79b06440effe None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:47.538505 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2f19c-738e-4dce-9e45-11d94b2c7bbc None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:47.601876 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3145bc2-4362-4c67-bbe2-79b06440effe None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:47.609555 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:47.623383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12124a1b-50a2-4c1a-9ad7-68827a45a228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:47.624099 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12124a1b-50a2-4c1a-9ad7-68827a45a228 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:47.624412 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12124a1b-50a2-4c1a-9ad7-68827a45a228 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:47.624689 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12124a1b-50a2-4c1a-9ad7-68827a45a228 None admin] PATH_INFO: `/v3/groups/da3fbc722b5249c6b875074fee5f3ef5` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:47.634339 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12124a1b-50a2-4c1a-9ad7-68827a45a228 None admin] RBAC: Authorizing `identity:get_group(group_id=da3fbc722b5249c6b875074fee5f3ef5)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:47.635852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12124a1b-50a2-4c1a-9ad7-68827a45a228 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:47.640130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 167/344] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Aug 6 12:17:47 2022] GET /identity/v3/groups/da3fbc722b5249c6b875074fee5f3ef5 => generated 226 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:47.641851 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc7f1b-2a58-4ab1-bd84-56b08a2f8471 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:47.642036 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc7f1b-2a58-4ab1-bd84-56b08a2f8471 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:47.642295 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc7f1b-2a58-4ab1-bd84-56b08a2f8471 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:47.643688 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 168/345] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:47.644804 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0ff95-10f5-462b-b4ce-f27cbe97aaee None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:47.645095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0ff95-10f5-462b-b4ce-f27cbe97aaee None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:47.645484 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0ff95-10f5-462b-b4ce-f27cbe97aaee None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:47.647004 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 169/346] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:47.651961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10124c-3424-4796-afe7-0ff8c27bbcbb None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:47.652250 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10124c-3424-4796-afe7-0ff8c27bbcbb None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:47.652589 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10124c-3424-4796-afe7-0ff8c27bbcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:47.661516 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3c8b6-7414-4771-a088-50504099d58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:47.662425 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3c8b6-7414-4771-a088-50504099d58b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:47.662940 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3c8b6-7414-4771-a088-50504099d58b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:47.663409 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3c8b6-7414-4771-a088-50504099d58b None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:47.664409 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3c8b6-7414-4771-a088-50504099d58b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:47.664927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3c8b6-7414-4771-a088-50504099d58b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:47.667258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3c8b6-7414-4771-a088-50504099d58b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:47.672513 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3b10124c-3424-4796-afe7-0ff8c27bbcbb None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:47.673824 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3b10124c-3424-4796-afe7-0ff8c27bbcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:47.686237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 178/347] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:47.687483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3c8b6-7414-4771-a088-50504099d58b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:47.720514 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:47.755390 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 170/348] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:47.757098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca8113-a403-45a6-863f-abe668d9e0ab None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:47.757564 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca8113-a403-45a6-863f-abe668d9e0ab None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:47.758050 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca8113-a403-45a6-863f-abe668d9e0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:47.786715 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-4aca8113-a403-45a6-863f-abe668d9e0ab None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:47.789732 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4aca8113-a403-45a6-863f-abe668d9e0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:47.844825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3c8b6-7414-4771-a088-50504099d58b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:47.882283 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 171/349] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:47.883057 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca8113-a403-45a6-863f-abe668d9e0ab None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:47.897556 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e307b762-e353-41ff-890d-f36e2c720f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:47.898215 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307b762-e353-41ff-890d-f36e2c720f2b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:47.898495 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307b762-e353-41ff-890d-f36e2c720f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:47.898770 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307b762-e353-41ff-890d-f36e2c720f2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:47.899567 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e307b762-e353-41ff-890d-f36e2c720f2b None admin] RBAC: Adding query filter params (group.id=da3fbc722b5249c6b875074fee5f3ef5, role.id=0fd7b75299b64c54971d9bfeaec30f4e, scope.project.id=5a84e3fab8334ecca0cd4aae83cb5317) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:47.899876 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e307b762-e353-41ff-890d-f36e2c720f2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:47.901911 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e307b762-e353-41ff-890d-f36e2c720f2b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:47.907906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 179/350] 10.222.0.20 () {56 vars in 1376 bytes} [Sat Aug 6 12:17:47 2022] GET /identity/v3/role_assignments?group.id=da3fbc722b5249c6b875074fee5f3ef5&scope.project.id=5a84e3fab8334ecca0cd4aae83cb5317&role.id=0fd7b75299b64c54971d9bfeaec30f4e => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:47.908652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e307b762-e353-41ff-890d-f36e2c720f2b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:47.971041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:48.026923 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca8113-a403-45a6-863f-abe668d9e0ab None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:48.038758 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:48.073817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e1ad0-e504-47de-bddc-eef08b5bf544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:48.074955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e1ad0-e504-47de-bddc-eef08b5bf544 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:48.075242 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e1ad0-e504-47de-bddc-eef08b5bf544 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:48.077020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e1ad0-e504-47de-bddc-eef08b5bf544 None admin] PATH_INFO: `/v3/users/e78afc5b30424635bc2e1f0a893c14ae` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:48.079507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e1ad0-e504-47de-bddc-eef08b5bf544 None admin] RBAC: Authorizing `identity:get_user(user_id=e78afc5b30424635bc2e1f0a893c14ae)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:48.083145 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e1ad0-e504-47de-bddc-eef08b5bf544 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:48.086038 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e307b762-e353-41ff-890d-f36e2c720f2b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:48.088128 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 172/351] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:17:47 2022] GET /identity/v3/users/e78afc5b30424635bc2e1f0a893c14ae => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:48.088998 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e1ad0-e504-47de-bddc-eef08b5bf544 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:48.154774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077ba35-d3d4-46c3-9056-7039409b09c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:48.155357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077ba35-d3d4-46c3-9056-7039409b09c2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:48.155638 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077ba35-d3d4-46c3-9056-7039409b09c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:48.155909 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077ba35-d3d4-46c3-9056-7039409b09c2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:48.174770 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077ba35-d3d4-46c3-9056-7039409b09c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:48.181808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-2077ba35-d3d4-46c3-9056-7039409b09c2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:17:48.181808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:48.181808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:48.181808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:48.181808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:48.181808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:48.181808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:48.181808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:48.181808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:48.181808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:48.181808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:48.181808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:48.181808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:48.181808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:48.181808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:48.181808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:48.182728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:48.182728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:48.182728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:48.182728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:48.182728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:48.182728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:48.182728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:48.182728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:48.182728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:48.182728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:48.182728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:48.182728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:48.182728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:48.182728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:48.182728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:48.182728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:48.183519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:48.183519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:48.183519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:48.183519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:48.183519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:48.183519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:48.183519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:48.183519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:48.183519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:48.183519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:48.183519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:17:48.183519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:48.183519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 180/352] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Aug 6 12:17:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:48.184669 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077ba35-d3d4-46c3-9056-7039409b09c2 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:48.240669 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e1ad0-e504-47de-bddc-eef08b5bf544 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:48.269133 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:48.283080 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4607bc-29de-46d6-8806-224c9b62010e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:48.283915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4607bc-29de-46d6-8806-224c9b62010e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:48.284266 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4607bc-29de-46d6-8806-224c9b62010e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:48.284630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4607bc-29de-46d6-8806-224c9b62010e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:48.296928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4607bc-29de-46d6-8806-224c9b62010e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:48.298283 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077ba35-d3d4-46c3-9056-7039409b09c2 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:48.302765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-4e4607bc-29de-46d6-8806-224c9b62010e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:48.302765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:48.302765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:48.302765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:48.302765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:48.302765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:48.302765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:48.302765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:48.302765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:48.302765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:48.302765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:48.302765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:48.302765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:48.302765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:48.302765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:48.302765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:48.303628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:48.303628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:48.303628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:48.303628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:48.303628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:48.303628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:48.303628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:48.303628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:48.303628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:48.303628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:48.303628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:48.303628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:48.303628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:48.303628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:48.303628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:48.303628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:48.304436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:48.304436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:48.304436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:48.304436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:48.304436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:48.304436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:48.304436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:48.304436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:48.304436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:48.304436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:48.304436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:48.304436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:17:48.304436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 173/353] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:48.305582 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4607bc-29de-46d6-8806-224c9b62010e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:48.335185 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0f572-ba3d-4f75-817e-b20ca9b9691e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:48.335918 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0f572-ba3d-4f75-817e-b20ca9b9691e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:48.336229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0f572-ba3d-4f75-817e-b20ca9b9691e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:48.336599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0f572-ba3d-4f75-817e-b20ca9b9691e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:48.337504 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0f572-ba3d-4f75-817e-b20ca9b9691e None admin] RBAC: Adding query filter params (role.id=7f46f07e73294f919cb56f7a97b324d7, user.id=e78afc5b30424635bc2e1f0a893c14ae, scope.system=all) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:48.337872 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0f572-ba3d-4f75-817e-b20ca9b9691e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:48.339418 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0f572-ba3d-4f75-817e-b20ca9b9691e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:48.360545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 181/354] 10.222.0.20 () {56 vars in 1308 bytes} [Sat Aug 6 12:17:48 2022] GET /identity/v3/role_assignments?user.id=e78afc5b30424635bc2e1f0a893c14ae&scope.system=all&role.id=7f46f07e73294f919cb56f7a97b324d7 => generated 510 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:48.362042 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d0488-eaa1-4ad2-9c75-e636d0d1623c None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:48.362332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d0488-eaa1-4ad2-9c75-e636d0d1623c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:48.362613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d0488-eaa1-4ad2-9c75-e636d0d1623c None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:48.364063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 182/355] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:48.364857 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d0488-eaa1-4ad2-9c75-e636d0d1623c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:48.372323 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:48.426808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4607bc-29de-46d6-8806-224c9b62010e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:48.471151 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2cbab9-9341-4da4-a1a9-9d5c87527cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:48.472058 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cbab9-9341-4da4-a1a9-9d5c87527cdd None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:48.472361 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cbab9-9341-4da4-a1a9-9d5c87527cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:48.473458 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cbab9-9341-4da4-a1a9-9d5c87527cdd None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:48.474206 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2cbab9-9341-4da4-a1a9-9d5c87527cdd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:48.474515 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2cbab9-9341-4da4-a1a9-9d5c87527cdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:48.476188 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2cbab9-9341-4da4-a1a9-9d5c87527cdd None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:48.507020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 174/356] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:17:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:48.508525 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b92b4-71cc-4423-8e9b-235307fbab78 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:48.508818 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b92b4-71cc-4423-8e9b-235307fbab78 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:48.509098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b92b4-71cc-4423-8e9b-235307fbab78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:48.547435 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-7a1b92b4-71cc-4423-8e9b-235307fbab78 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:48.548692 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7a1b92b4-71cc-4423-8e9b-235307fbab78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:48.570530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d0488-eaa1-4ad2-9c75-e636d0d1623c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:48.646247 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:48.651994 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d186b-49bb-4d5b-af28-d9c9c947415d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:48.652815 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d186b-49bb-4d5b-af28-d9c9c947415d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:48.653294 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d186b-49bb-4d5b-af28-d9c9c947415d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:48.653576 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d186b-49bb-4d5b-af28-d9c9c947415d None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:48.654329 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d186b-49bb-4d5b-af28-d9c9c947415d None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:48.654634 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d186b-49bb-4d5b-af28-d9c9c947415d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:48.656155 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d186b-49bb-4d5b-af28-d9c9c947415d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:48.667042 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 183/357] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:48.668015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d186b-49bb-4d5b-af28-d9c9c947415d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:48.682621 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 175/358] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:48.688065 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0902d-9f28-4623-b64c-8c59c536d561 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:48.688377 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0902d-9f28-4623-b64c-8c59c536d561 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:48.688656 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0902d-9f28-4623-b64c-8c59c536d561 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:48.690389 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 176/359] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:48.691128 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0902d-9f28-4623-b64c-8c59c536d561 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:48.743083 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:48.782909 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d186b-49bb-4d5b-af28-d9c9c947415d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:48.819931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0902d-9f28-4623-b64c-8c59c536d561 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:48.823882 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1b3b2-4c06-4c27-aa1d-33ef49a31fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:48.824572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1b3b2-4c06-4c27-aa1d-33ef49a31fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:48.824853 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1b3b2-4c06-4c27-aa1d-33ef49a31fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:48.825121 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1b3b2-4c06-4c27-aa1d-33ef49a31fd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:48.859449 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4255cfb-7c42-4ad7-a98c-b00e3eb0ae71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:48.860077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4255cfb-7c42-4ad7-a98c-b00e3eb0ae71 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:48.860390 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4255cfb-7c42-4ad7-a98c-b00e3eb0ae71 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:48.860666 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4255cfb-7c42-4ad7-a98c-b00e3eb0ae71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:48.912217 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1b3b2-4c06-4c27-aa1d-33ef49a31fd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:48.913719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1b3b2-4c06-4c27-aa1d-33ef49a31fd7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:48.922715 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:48.957389 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4255cfb-7c42-4ad7-a98c-b00e3eb0ae71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:48.959164 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4255cfb-7c42-4ad7-a98c-b00e3eb0ae71 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:48.995421 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:49.043708 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 184/360] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:49.045331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e956e-9b65-4e64-b608-4e486fd739a8 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:49.045616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e956e-9b65-4e64-b608-4e486fd739a8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:49.045888 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e956e-9b65-4e64-b608-4e486fd739a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:49.066286 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-522e956e-9b65-4e64-b608-4e486fd739a8 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:49.067487 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-522e956e-9b65-4e64-b608-4e486fd739a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:49.116462 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 177/361] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:49.117554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4255cfb-7c42-4ad7-a98c-b00e3eb0ae71 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:49.151835 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 185/362] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:49.152721 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e956e-9b65-4e64-b608-4e486fd739a8 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:49.213254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:49.246570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4255cfb-7c42-4ad7-a98c-b00e3eb0ae71 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:49.283115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:49.291753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e956e-9b65-4e64-b608-4e486fd739a8 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:49.294038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431eebf8-4568-411e-a78a-b299c2063688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:49.294710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431eebf8-4568-411e-a78a-b299c2063688 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:49.294991 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431eebf8-4568-411e-a78a-b299c2063688 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:49.295266 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431eebf8-4568-411e-a78a-b299c2063688 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:49.309581 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431eebf8-4568-411e-a78a-b299c2063688 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:49.314996 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-431eebf8-4568-411e-a78a-b299c2063688 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:17:49.314996 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:49.314996 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:49.314996 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:49.314996 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:49.314996 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:49.314996 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:49.314996 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:49.314996 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:49.314996 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:49.314996 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:49.314996 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:49.314996 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:49.314996 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:49.314996 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:49.314996 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:49.315896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:49.315896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:49.315896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:49.315896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:49.315896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:49.315896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:49.315896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:49.315896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:49.315896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:49.315896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:49.315896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:49.315896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:49.315896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:49.315896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:49.315896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:49.315896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:49.316708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:49.316708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:49.316708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:49.316708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:49.316708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:49.316708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:49.316708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:49.316708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:49.316708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:49.316708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:49.316708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:17:49.316708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:17:49.316708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 178/363] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Aug 6 12:17:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:49.318041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431eebf8-4568-411e-a78a-b299c2063688 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:49.334913 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110dcad9-9bd3-4b70-8548-ee9b0a9aab59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:49.335829 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dcad9-9bd3-4b70-8548-ee9b0a9aab59 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:49.336352 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dcad9-9bd3-4b70-8548-ee9b0a9aab59 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:49.336819 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dcad9-9bd3-4b70-8548-ee9b0a9aab59 None admin] PATH_INFO: `/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:49.338773 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110dcad9-9bd3-4b70-8548-ee9b0a9aab59 None admin] RBAC: Authorizing `identity:get_project(project_id=5a84e3fab8334ecca0cd4aae83cb5317)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:49.340520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110dcad9-9bd3-4b70-8548-ee9b0a9aab59 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:49.342516 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 186/364] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:49 2022] GET /identity/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:49.344239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110dcad9-9bd3-4b70-8548-ee9b0a9aab59 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:49.446488 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431eebf8-4568-411e-a78a-b299c2063688 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:49.474221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110dcad9-9bd3-4b70-8548-ee9b0a9aab59 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:49.487994 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c4eb1-1b1d-4484-8cc5-51e9d28ba0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:49.488872 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c4eb1-1b1d-4484-8cc5-51e9d28ba0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:49.489289 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c4eb1-1b1d-4484-8cc5-51e9d28ba0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:49.489773 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c4eb1-1b1d-4484-8cc5-51e9d28ba0c4 None admin] PATH_INFO: `/v3/projects/34ca99f1da904d608e324f65f6571250` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:49.491330 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c4eb1-1b1d-4484-8cc5-51e9d28ba0c4 None admin] RBAC: Authorizing `identity:get_project(project_id=34ca99f1da904d608e324f65f6571250)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:49.492934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c4eb1-1b1d-4484-8cc5-51e9d28ba0c4 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:49.495276 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 179/365] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:49 2022] GET /identity/v3/projects/34ca99f1da904d608e324f65f6571250 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:49.499799 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c4eb1-1b1d-4484-8cc5-51e9d28ba0c4 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:49.502204 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:49.523917 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096f7f4-a77e-410a-a745-0c562dbe16c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:49.524885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096f7f4-a77e-410a-a745-0c562dbe16c1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:49.525206 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096f7f4-a77e-410a-a745-0c562dbe16c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:49.525479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096f7f4-a77e-410a-a745-0c562dbe16c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:49.537520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096f7f4-a77e-410a-a745-0c562dbe16c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:49.543049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-2096f7f4-a77e-410a-a745-0c562dbe16c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:17:49.543049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:49.543049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:49.543049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:49.543049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:49.543049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:49.543049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:49.543049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:49.543049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:49.543049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:49.543049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:49.543049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:49.543049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:49.543049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:49.543049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:49.543049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:49.544067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:49.544067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:49.544067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:49.544067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:49.544067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:49.544067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:49.544067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:49.544067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:49.544067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:49.544067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:49.544067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:49.544067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:49.544067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:49.544067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:49.544067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:49.544067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:49.545048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:49.545048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:49.545048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:49.545048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:49.545048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:49.545048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:49.545048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:49.545048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:49.545048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:49.545048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:49.545048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:17:49.545048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:49.545048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 187/366] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:17:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:49.545048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096f7f4-a77e-410a-a745-0c562dbe16c1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:49.580433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:49.639015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c4eb1-1b1d-4484-8cc5-51e9d28ba0c4 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:49.667205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096f7f4-a77e-410a-a745-0c562dbe16c1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:49.685954 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b651660-979a-4dd2-830c-8b5db6b49e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:49.687118 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b651660-979a-4dd2-830c-8b5db6b49e8b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:49.687543 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b651660-979a-4dd2-830c-8b5db6b49e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:49.688128 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b651660-979a-4dd2-830c-8b5db6b49e8b None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:49.689451 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b651660-979a-4dd2-830c-8b5db6b49e8b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:49.689978 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b651660-979a-4dd2-830c-8b5db6b49e8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:49.691511 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b651660-979a-4dd2-830c-8b5db6b49e8b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:49.701933 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 180/367] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:17:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:49.702969 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b651660-979a-4dd2-830c-8b5db6b49e8b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:49.717508 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371a9b2-cbca-486e-9476-0e34bdd6ad60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:49.717744 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371a9b2-cbca-486e-9476-0e34bdd6ad60 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:49.717995 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371a9b2-cbca-486e-9476-0e34bdd6ad60 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:49.718265 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371a9b2-cbca-486e-9476-0e34bdd6ad60 None admin] PATH_INFO: `/v3/groups/da3fbc722b5249c6b875074fee5f3ef5` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:49.719593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371a9b2-cbca-486e-9476-0e34bdd6ad60 None admin] RBAC: Authorizing `identity:get_group(group_id=da3fbc722b5249c6b875074fee5f3ef5)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:49.721200 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371a9b2-cbca-486e-9476-0e34bdd6ad60 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:49.722908 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 188/368] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Aug 6 12:17:49 2022] GET /identity/v3/groups/da3fbc722b5249c6b875074fee5f3ef5 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:49.723675 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371a9b2-cbca-486e-9476-0e34bdd6ad60 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:49.776156 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:49.827604 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b651660-979a-4dd2-830c-8b5db6b49e8b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:49.834898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:49.844888 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371a9b2-cbca-486e-9476-0e34bdd6ad60 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:49.867366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf262c8-5b22-4626-bc2d-3d4a6793325d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:49.868402 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf262c8-5b22-4626-bc2d-3d4a6793325d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:49.869402 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf262c8-5b22-4626-bc2d-3d4a6793325d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:49.869881 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf262c8-5b22-4626-bc2d-3d4a6793325d None admin] PATH_INFO: `/v3/groups/da3fbc722b5249c6b875074fee5f3ef5` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:49.871575 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf262c8-5b22-4626-bc2d-3d4a6793325d None admin] RBAC: Authorizing `identity:get_group(group_id=da3fbc722b5249c6b875074fee5f3ef5)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:49.873340 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf262c8-5b22-4626-bc2d-3d4a6793325d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:49.875367 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 181/369] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Aug 6 12:17:49 2022] GET /identity/v3/groups/da3fbc722b5249c6b875074fee5f3ef5 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:49.876361 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf262c8-5b22-4626-bc2d-3d4a6793325d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:49.882425 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd558a-4977-4ed1-bee4-4c19269e0ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:49.883042 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd558a-4977-4ed1-bee4-4c19269e0ecf None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:49.883317 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd558a-4977-4ed1-bee4-4c19269e0ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:49.883588 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefd558a-4977-4ed1-bee4-4c19269e0ecf None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:49.884338 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefd558a-4977-4ed1-bee4-4c19269e0ecf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:49.885255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefd558a-4977-4ed1-bee4-4c19269e0ecf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:49.886600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefd558a-4977-4ed1-bee4-4c19269e0ecf None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:49.895591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 189/370] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:49.896788 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd558a-4977-4ed1-bee4-4c19269e0ecf None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:50.014257 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf262c8-5b22-4626-bc2d-3d4a6793325d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:50.023291 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefd558a-4977-4ed1-bee4-4c19269e0ecf None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:50.067452 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:50.070246 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7e159-495e-4882-83cb-d50437fc2037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:50.070921 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7e159-495e-4882-83cb-d50437fc2037 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:50.071154 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da283b-1f3c-4ff7-a4d3-96ef42cc3185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:50.071653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7e159-495e-4882-83cb-d50437fc2037 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:50.071782 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da283b-1f3c-4ff7-a4d3-96ef42cc3185 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:50.072112 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da283b-1f3c-4ff7-a4d3-96ef42cc3185 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:50.072423 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da283b-1f3c-4ff7-a4d3-96ef42cc3185 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:50.072730 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7e159-495e-4882-83cb-d50437fc2037 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:50.073216 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da283b-1f3c-4ff7-a4d3-96ef42cc3185 None admin] RBAC: Adding query filter params (group.id=da3fbc722b5249c6b875074fee5f3ef5, role.id=47cde553d7ee465bbe685a05c00b94e2, scope.project.id=5a84e3fab8334ecca0cd4aae83cb5317) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:50.073594 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da283b-1f3c-4ff7-a4d3-96ef42cc3185 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:50.075030 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3da283b-1f3c-4ff7-a4d3-96ef42cc3185 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:50.081882 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 190/371] 10.222.0.20 () {56 vars in 1376 bytes} [Sat Aug 6 12:17:49 2022] GET /identity/v3/role_assignments?group.id=da3fbc722b5249c6b875074fee5f3ef5&scope.project.id=5a84e3fab8334ecca0cd4aae83cb5317&role.id=47cde553d7ee465bbe685a05c00b94e2 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:50.082685 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da283b-1f3c-4ff7-a4d3-96ef42cc3185 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:50.110064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:50.158262 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7e159-495e-4882-83cb-d50437fc2037 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:50.159749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7e159-495e-4882-83cb-d50437fc2037 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:50.209428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3da283b-1f3c-4ff7-a4d3-96ef42cc3185 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:50.248502 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970b1da-3887-4241-baf1-50a9ede06212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:50.249136 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970b1da-3887-4241-baf1-50a9ede06212 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:50.249421 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970b1da-3887-4241-baf1-50a9ede06212 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:50.249942 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970b1da-3887-4241-baf1-50a9ede06212 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:50.250817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0970b1da-3887-4241-baf1-50a9ede06212 None admin] RBAC: Adding query filter params (group.id=da3fbc722b5249c6b875074fee5f3ef5, role.id=0fd7b75299b64c54971d9bfeaec30f4e, scope.project.id=34ca99f1da904d608e324f65f6571250) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:50.251132 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0970b1da-3887-4241-baf1-50a9ede06212 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:50.252624 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0970b1da-3887-4241-baf1-50a9ede06212 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:50.259181 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 191/372] 10.222.0.20 () {56 vars in 1376 bytes} [Sat Aug 6 12:17:50 2022] GET /identity/v3/role_assignments?group.id=da3fbc722b5249c6b875074fee5f3ef5&scope.project.id=34ca99f1da904d608e324f65f6571250&role.id=0fd7b75299b64c54971d9bfeaec30f4e => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:50.260721 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970b1da-3887-4241-baf1-50a9ede06212 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:50.287772 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 182/373] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:50.289620 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e326a-7195-46e4-aed1-3e6ff224f216 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:50.290108 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e326a-7195-46e4-aed1-3e6ff224f216 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:50.290705 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e326a-7195-46e4-aed1-3e6ff224f216 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:50.292857 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 183/374] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:50.301558 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a873d-c47d-4a29-8477-fcd3490cd4b4 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:50.302176 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a873d-c47d-4a29-8477-fcd3490cd4b4 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:50.302653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a873d-c47d-4a29-8477-fcd3490cd4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:50.330694 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-e17a873d-c47d-4a29-8477-fcd3490cd4b4 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:50.332221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e17a873d-c47d-4a29-8477-fcd3490cd4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:50.368417 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:50.380777 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:50.408816 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970b1da-3887-4241-baf1-50a9ede06212 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:50.431176 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 184/375] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:50.433493 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a873d-c47d-4a29-8477-fcd3490cd4b4 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:50.459148 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e855d90-55f1-42a5-a462-d538e031fb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:50.459781 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e855d90-55f1-42a5-a462-d538e031fb57 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:50.460060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e855d90-55f1-42a5-a462-d538e031fb57 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:50.460421 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e855d90-55f1-42a5-a462-d538e031fb57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:50.563449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e855d90-55f1-42a5-a462-d538e031fb57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:50.564998 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e855d90-55f1-42a5-a462-d538e031fb57 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:50.568458 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a873d-c47d-4a29-8477-fcd3490cd4b4 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:50.613003 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be503f1f-5ff8-48c8-8750-aeb5a2272086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:50.613998 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be503f1f-5ff8-48c8-8750-aeb5a2272086 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:50.614848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be503f1f-5ff8-48c8-8750-aeb5a2272086 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:50.615302 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be503f1f-5ff8-48c8-8750-aeb5a2272086 None admin] PATH_INFO: `/v3/projects/34ca99f1da904d608e324f65f6571250` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:50.617568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be503f1f-5ff8-48c8-8750-aeb5a2272086 None admin] RBAC: Authorizing `identity:get_project(project_id=34ca99f1da904d608e324f65f6571250)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:50.620008 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be503f1f-5ff8-48c8-8750-aeb5a2272086 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:50.622812 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 185/376] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:50 2022] GET /identity/v3/projects/34ca99f1da904d608e324f65f6571250 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:50.623806 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be503f1f-5ff8-48c8-8750-aeb5a2272086 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:50.662128 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:50.680709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:50.701486 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 192/377] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:50.702361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e855d90-55f1-42a5-a462-d538e031fb57 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:50.766434 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be503f1f-5ff8-48c8-8750-aeb5a2272086 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:50.811443 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b197ab-c6ed-42f1-a4c0-51553e34f51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:50.812591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b197ab-c6ed-42f1-a4c0-51553e34f51a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:50.813085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b197ab-c6ed-42f1-a4c0-51553e34f51a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:50.813570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b197ab-c6ed-42f1-a4c0-51553e34f51a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:50.827725 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e855d90-55f1-42a5-a462-d538e031fb57 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:50.828423 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b197ab-c6ed-42f1-a4c0-51553e34f51a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:50.834062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-22b197ab-c6ed-42f1-a4c0-51553e34f51a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:50.834062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:50.834062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:50.834062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:50.834062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:50.834062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:50.834062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:50.834062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:50.834062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:50.834062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:50.834062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:50.834062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:50.834062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:50.834062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:50.834062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:50.834062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:50.834961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:50.834961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:50.834961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:50.834961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:50.834961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:50.834961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:50.834961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:50.834961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:50.834961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:50.834961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:50.834961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:50.834961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:50.834961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:50.834961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:50.834961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:50.834961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:50.835744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:50.835744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:50.835744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:50.835744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:50.835744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:50.835744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:50.835744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:50.835744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:50.835744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:50.835744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:50.835744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:50.835744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:17:50.839185 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 186/378] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:50.840117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b197ab-c6ed-42f1-a4c0-51553e34f51a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:50.868392 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316e3e9-4fdd-47ce-b5f9-b680f834437d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:50.869010 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316e3e9-4fdd-47ce-b5f9-b680f834437d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:50.869292 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316e3e9-4fdd-47ce-b5f9-b680f834437d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:50.869561 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316e3e9-4fdd-47ce-b5f9-b680f834437d None admin] PATH_INFO: `/v3/groups/da3fbc722b5249c6b875074fee5f3ef5` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:50.871057 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316e3e9-4fdd-47ce-b5f9-b680f834437d None admin] RBAC: Authorizing `identity:get_group(group_id=da3fbc722b5249c6b875074fee5f3ef5)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:50.876539 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316e3e9-4fdd-47ce-b5f9-b680f834437d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:50.878695 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 193/379] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Aug 6 12:17:50 2022] GET /identity/v3/groups/da3fbc722b5249c6b875074fee5f3ef5 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:50.879470 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316e3e9-4fdd-47ce-b5f9-b680f834437d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:50.938956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:50.944094 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:50.958968 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b197ab-c6ed-42f1-a4c0-51553e34f51a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:51.017655 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316e3e9-4fdd-47ce-b5f9-b680f834437d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:51.018836 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90483422-3ff6-4c09-ba22-4e00aa224fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:51.019882 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90483422-3ff6-4c09-ba22-4e00aa224fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:51.020434 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90483422-3ff6-4c09-ba22-4e00aa224fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:51.021785 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90483422-3ff6-4c09-ba22-4e00aa224fc9 None admin] PATH_INFO: `/v3/projects/7ad154ab2bd64d7e8e712fcec9e1c94f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:51.023340 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90483422-3ff6-4c09-ba22-4e00aa224fc9 None admin] RBAC: Authorizing `identity:get_project(project_id=7ad154ab2bd64d7e8e712fcec9e1c94f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:51.025169 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90483422-3ff6-4c09-ba22-4e00aa224fc9 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:51.027832 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 187/380] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:50 2022] GET /identity/v3/projects/7ad154ab2bd64d7e8e712fcec9e1c94f => generated 344 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:51.028955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90483422-3ff6-4c09-ba22-4e00aa224fc9 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:51.066387 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004142dc-d503-43a6-9a94-a6eae450af21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:51.067611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004142dc-d503-43a6-9a94-a6eae450af21 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:51.068525 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004142dc-d503-43a6-9a94-a6eae450af21 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:51.069172 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004142dc-d503-43a6-9a94-a6eae450af21 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:51.070234 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004142dc-d503-43a6-9a94-a6eae450af21 None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:51.070994 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004142dc-d503-43a6-9a94-a6eae450af21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:51.073786 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004142dc-d503-43a6-9a94-a6eae450af21 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:51.084819 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 194/381] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:51.086490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004142dc-d503-43a6-9a94-a6eae450af21 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:51.171011 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90483422-3ff6-4c09-ba22-4e00aa224fc9 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:51.211267 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004142dc-d503-43a6-9a94-a6eae450af21 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:51.218217 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66981cf-0ff2-4df5-bd9f-c74d6a230777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:51.219086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66981cf-0ff2-4df5-bd9f-c74d6a230777 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:51.219605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66981cf-0ff2-4df5-bd9f-c74d6a230777 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:51.220109 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66981cf-0ff2-4df5-bd9f-c74d6a230777 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:51.221239 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66981cf-0ff2-4df5-bd9f-c74d6a230777 None admin] RBAC: Adding query filter params (group.id=da3fbc722b5249c6b875074fee5f3ef5, role.id=47cde553d7ee465bbe685a05c00b94e2, scope.project.id=34ca99f1da904d608e324f65f6571250) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:51.221552 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66981cf-0ff2-4df5-bd9f-c74d6a230777 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:51.222988 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66981cf-0ff2-4df5-bd9f-c74d6a230777 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:51.229022 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 188/382] 10.222.0.20 () {56 vars in 1376 bytes} [Sat Aug 6 12:17:51 2022] GET /identity/v3/role_assignments?group.id=da3fbc722b5249c6b875074fee5f3ef5&scope.project.id=34ca99f1da904d608e324f65f6571250&role.id=47cde553d7ee465bbe685a05c00b94e2 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:51.232038 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:51.237182 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66981cf-0ff2-4df5-bd9f-c74d6a230777 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:51.262349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f445a-040f-4e59-a7a9-2aea7dd14510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:51.263446 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f445a-040f-4e59-a7a9-2aea7dd14510 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:51.263932 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f445a-040f-4e59-a7a9-2aea7dd14510 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:51.264438 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f445a-040f-4e59-a7a9-2aea7dd14510 None admin] PATH_INFO: `/v3/groups/d78e995107bc436bb56514be0b1f1a06` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:51.272634 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f445a-040f-4e59-a7a9-2aea7dd14510 None admin] RBAC: Authorizing `identity:get_group(group_id=d78e995107bc436bb56514be0b1f1a06)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:51.275561 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f445a-040f-4e59-a7a9-2aea7dd14510 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:51.281869 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 195/383] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Aug 6 12:17:51 2022] GET /identity/v3/groups/d78e995107bc436bb56514be0b1f1a06 => generated 229 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:51.285791 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f445a-040f-4e59-a7a9-2aea7dd14510 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:51.292172 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:51.415544 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f445a-040f-4e59-a7a9-2aea7dd14510 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:51.473469 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afcf98-01c1-4dc3-b07c-46c7af1816bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:51.474729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afcf98-01c1-4dc3-b07c-46c7af1816bc None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:51.475231 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afcf98-01c1-4dc3-b07c-46c7af1816bc None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:51.475716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afcf98-01c1-4dc3-b07c-46c7af1816bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:51.476853 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8afcf98-01c1-4dc3-b07c-46c7af1816bc None admin] RBAC: Adding query filter params (group.id=d78e995107bc436bb56514be0b1f1a06, role.id=144c8425688b409384e1e8ed8279d88e, scope.project.id=7ad154ab2bd64d7e8e712fcec9e1c94f) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:51.477963 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8afcf98-01c1-4dc3-b07c-46c7af1816bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:51.480004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8afcf98-01c1-4dc3-b07c-46c7af1816bc None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:51.490177 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 196/384] 10.222.0.20 () {56 vars in 1376 bytes} [Sat Aug 6 12:17:51 2022] GET /identity/v3/role_assignments?group.id=d78e995107bc436bb56514be0b1f1a06&scope.project.id=7ad154ab2bd64d7e8e712fcec9e1c94f&role.id=144c8425688b409384e1e8ed8279d88e => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:51.505368 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66981cf-0ff2-4df5-bd9f-c74d6a230777 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:51.565661 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97731b-975e-4f41-89e2-8dc2cd4d359b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:51.566319 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97731b-975e-4f41-89e2-8dc2cd4d359b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:51.566598 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97731b-975e-4f41-89e2-8dc2cd4d359b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:51.566873 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97731b-975e-4f41-89e2-8dc2cd4d359b None admin] PATH_INFO: `/v3/groups/da3fbc722b5249c6b875074fee5f3ef5` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:51.568033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97731b-975e-4f41-89e2-8dc2cd4d359b None admin] RBAC: Authorizing `identity:get_group(group_id=da3fbc722b5249c6b875074fee5f3ef5)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:51.569530 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97731b-975e-4f41-89e2-8dc2cd4d359b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:51.571408 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 189/385] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Aug 6 12:17:51 2022] GET /identity/v3/groups/da3fbc722b5249c6b875074fee5f3ef5 => generated 226 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:51.577659 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afcf98-01c1-4dc3-b07c-46c7af1816bc None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:51.637749 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:51.707995 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afcf98-01c1-4dc3-b07c-46c7af1816bc None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:51.748662 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c462da-0dd3-433f-be83-f5bb1ea26203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:51.749656 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c462da-0dd3-433f-be83-f5bb1ea26203 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:51.750139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c462da-0dd3-433f-be83-f5bb1ea26203 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:51.750617 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c462da-0dd3-433f-be83-f5bb1ea26203 None admin] PATH_INFO: `/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:51.752039 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c462da-0dd3-433f-be83-f5bb1ea26203 None admin] RBAC: Authorizing `identity:get_project(project_id=5a84e3fab8334ecca0cd4aae83cb5317)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:51.753771 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c462da-0dd3-433f-be83-f5bb1ea26203 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:51.755763 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 197/386] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:51 2022] GET /identity/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:51.762207 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97731b-975e-4f41-89e2-8dc2cd4d359b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:51.886759 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:51.898119 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97731b-975e-4f41-89e2-8dc2cd4d359b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:51.950009 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a507931b-0e71-4927-a2c5-87c10e0bd3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:51.950691 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507931b-0e71-4927-a2c5-87c10e0bd3a0 None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:51.950972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507931b-0e71-4927-a2c5-87c10e0bd3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:51.951246 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a507931b-0e71-4927-a2c5-87c10e0bd3a0 None admin] PATH_INFO: `/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317/groups/da3fbc722b5249c6b875074fee5f3ef5/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:51.953254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a507931b-0e71-4927-a2c5-87c10e0bd3a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a84e3fab8334ecca0cd4aae83cb5317, group_id=da3fbc722b5249c6b875074fee5f3ef5, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:51.954693 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a507931b-0e71-4927-a2c5-87c10e0bd3a0 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:51.974476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 190/387] 10.222.0.20 () {58 vars in 1442 bytes} [Sat Aug 6 12:17:51 2022] PUT /identity/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317/groups/da3fbc722b5249c6b875074fee5f3ef5/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:17:52.597888 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c79a8-f579-4a23-9c2b-bcf0ce49ee15 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:52.598912 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c79a8-f579-4a23-9c2b-bcf0ce49ee15 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:52.599436 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c79a8-f579-4a23-9c2b-bcf0ce49ee15 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:52.601339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 198/388] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:52.605248 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a1d67-ac87-4c0e-8bcf-30ff881ea5ac None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:52.605659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a1d67-ac87-4c0e-8bcf-30ff881ea5ac None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:52.605864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a1d67-ac87-4c0e-8bcf-30ff881ea5ac None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:52.608740 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026df756-11fd-4a02-8449-ea62b0f83e37 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:52.609020 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026df756-11fd-4a02-8449-ea62b0f83e37 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:52.609292 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026df756-11fd-4a02-8449-ea62b0f83e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:52.610904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 191/389] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:52.625233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f3af6-4dce-43f4-8bb3-375b27172418 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:52.625542 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f3af6-4dce-43f4-8bb3-375b27172418 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:52.625776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f3af6-4dce-43f4-8bb3-375b27172418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:52.652103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-815f3af6-4dce-43f4-8bb3-375b27172418 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:52.653306 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-026df756-11fd-4a02-8449-ea62b0f83e37 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:52.653418 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-815f3af6-4dce-43f4-8bb3-375b27172418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:52.654488 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-026df756-11fd-4a02-8449-ea62b0f83e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:52.752592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 199/390] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:52.771879 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 192/391] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 12:17:52.777531 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026df756-11fd-4a02-8449-ea62b0f83e37 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:52.806386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f3af6-4dce-43f4-8bb3-375b27172418 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:52.860727 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:52.911184 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026df756-11fd-4a02-8449-ea62b0f83e37 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:52.940694 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:52.953310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f651a-1ff0-49ba-8ae9-05d0d43099bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:52.954222 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f651a-1ff0-49ba-8ae9-05d0d43099bc None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:52.954717 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f651a-1ff0-49ba-8ae9-05d0d43099bc None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:52.955214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f651a-1ff0-49ba-8ae9-05d0d43099bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:52.976365 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f3af6-4dce-43f4-8bb3-375b27172418 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:52.977553 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f651a-1ff0-49ba-8ae9-05d0d43099bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:52.983331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-492f651a-1ff0-49ba-8ae9-05d0d43099bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:17:52.983331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:52.983331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:52.983331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:52.983331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:52.983331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:52.983331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:52.983331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:52.983331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:52.983331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:52.983331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:52.983331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:52.983331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:52.983331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:52.983331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:52.983331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:52.984195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:52.984195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:52.984195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:52.984195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:52.984195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:52.984195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:52.984195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:52.984195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:52.984195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:52.984195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:52.984195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:52.984195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:52.984195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:52.984195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:52.984195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:52.984195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:52.985005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:52.985005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:52.985005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:52.985005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:52.985005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:52.985005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:52.985005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:52.985005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:52.985005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:52.985005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:52.985005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:17:52.985005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:52.986924 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 200/392] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Aug 6 12:17:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:53.009612 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f651a-1ff0-49ba-8ae9-05d0d43099bc None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:53.017908 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a41a76-c181-465d-abc0-cc2217a7cea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:53.018698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a41a76-c181-465d-abc0-cc2217a7cea3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:53.019072 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a41a76-c181-465d-abc0-cc2217a7cea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:53.019442 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a41a76-c181-465d-abc0-cc2217a7cea3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:53.032054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a41a76-c181-465d-abc0-cc2217a7cea3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:53.037332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-77a41a76-c181-465d-abc0-cc2217a7cea3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:53.037332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:53.037332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:53.037332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:53.037332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:53.037332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:53.037332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:53.037332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:53.037332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:53.037332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:53.037332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:53.037332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:53.037332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:53.037332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:53.037332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:53.037332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:53.038409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:53.038409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:53.038409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:53.038409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:53.038409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:53.038409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:53.038409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:53.038409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:53.038409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:53.038409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:53.038409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:53.038409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:53.038409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:53.038409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:53.038409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:53.038409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:53.039409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:53.039409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:53.039409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:53.039409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:53.039409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:53.039409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:53.039409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:53.039409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:53.039409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:53.039409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:53.039409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:53.039409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:17:53.048949 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 193/393] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:17:53.049638 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a41a76-c181-465d-abc0-cc2217a7cea3 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:53.139605 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:53.145230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f651a-1ff0-49ba-8ae9-05d0d43099bc None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:53.182695 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e33ba1-2f9a-47e6-b0f1-2b75455a12fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:53.183671 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e33ba1-2f9a-47e6-b0f1-2b75455a12fd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:53.184166 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e33ba1-2f9a-47e6-b0f1-2b75455a12fd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:53.184651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e33ba1-2f9a-47e6-b0f1-2b75455a12fd None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:53.185562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e33ba1-2f9a-47e6-b0f1-2b75455a12fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:53.186089 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e33ba1-2f9a-47e6-b0f1-2b75455a12fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:53.187808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e33ba1-2f9a-47e6-b0f1-2b75455a12fd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:53.196685 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 201/394] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:17:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:53.202246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e33ba1-2f9a-47e6-b0f1-2b75455a12fd None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:53.302451 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a41a76-c181-465d-abc0-cc2217a7cea3 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:53.306490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e33ba1-2f9a-47e6-b0f1-2b75455a12fd None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:53.341166 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9a61c-ffdd-4ba0-8183-07100e716c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:53.342120 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9a61c-ffdd-4ba0-8183-07100e716c4c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:53.342590 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9a61c-ffdd-4ba0-8183-07100e716c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:53.343052 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9a61c-ffdd-4ba0-8183-07100e716c4c None admin] PATH_INFO: `/v3/groups/da3fbc722b5249c6b875074fee5f3ef5` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:53.344628 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9a61c-ffdd-4ba0-8183-07100e716c4c None admin] RBAC: Authorizing `identity:get_group(group_id=da3fbc722b5249c6b875074fee5f3ef5)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:53.346229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9a61c-ffdd-4ba0-8183-07100e716c4c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:53.348066 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 202/395] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Aug 6 12:17:53 2022] GET /identity/v3/groups/da3fbc722b5249c6b875074fee5f3ef5 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:53.361438 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9a61c-ffdd-4ba0-8183-07100e716c4c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:53.368535 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea242e69-56c8-4e27-9a40-ee9ee0be1409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:53.369259 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea242e69-56c8-4e27-9a40-ee9ee0be1409 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:53.369547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea242e69-56c8-4e27-9a40-ee9ee0be1409 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:53.369822 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea242e69-56c8-4e27-9a40-ee9ee0be1409 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:53.370540 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea242e69-56c8-4e27-9a40-ee9ee0be1409 None admin] RBAC: Adding query filter params (name=member) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:53.370845 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea242e69-56c8-4e27-9a40-ee9ee0be1409 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:53.378094 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea242e69-56c8-4e27-9a40-ee9ee0be1409 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:53.382862 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 194/396] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:53.390361 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea242e69-56c8-4e27-9a40-ee9ee0be1409 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:53.401693 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:53.418756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:53.500374 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea242e69-56c8-4e27-9a40-ee9ee0be1409 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:53.521424 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9a61c-ffdd-4ba0-8183-07100e716c4c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:53.545102 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b20c08-d186-455a-91a6-98e520782d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:53.545751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b20c08-d186-455a-91a6-98e520782d93 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:53.546035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b20c08-d186-455a-91a6-98e520782d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:53.546311 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b20c08-d186-455a-91a6-98e520782d93 None admin] PATH_INFO: `/v3/groups/da3fbc722b5249c6b875074fee5f3ef5` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:53.547480 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b20c08-d186-455a-91a6-98e520782d93 None admin] RBAC: Authorizing `identity:get_group(group_id=da3fbc722b5249c6b875074fee5f3ef5)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:53.548976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b20c08-d186-455a-91a6-98e520782d93 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:53.550762 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 195/397] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Aug 6 12:17:53 2022] GET /identity/v3/groups/da3fbc722b5249c6b875074fee5f3ef5 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:53.557572 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b20c08-d186-455a-91a6-98e520782d93 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:53.594901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31023445-59f4-470b-9784-c1fa13e0e5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:53.595554 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31023445-59f4-470b-9784-c1fa13e0e5ad None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:53.595837 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31023445-59f4-470b-9784-c1fa13e0e5ad None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:53.596111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31023445-59f4-470b-9784-c1fa13e0e5ad None admin] PATH_INFO: `/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:53.600751 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31023445-59f4-470b-9784-c1fa13e0e5ad None admin] RBAC: Authorizing `identity:get_project(project_id=5a84e3fab8334ecca0cd4aae83cb5317)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:53.602210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31023445-59f4-470b-9784-c1fa13e0e5ad None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:53.604502 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 203/398] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:53 2022] GET /identity/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317 => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:53.610277 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31023445-59f4-470b-9784-c1fa13e0e5ad None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:53.699790 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:53.736445 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b20c08-d186-455a-91a6-98e520782d93 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:53.771392 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:53.773653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619257b-7aae-479a-9bf6-361ddf6dccb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:53.774411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619257b-7aae-479a-9bf6-361ddf6dccb0 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:53.774843 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619257b-7aae-479a-9bf6-361ddf6dccb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:53.775754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619257b-7aae-479a-9bf6-361ddf6dccb0 None admin] PATH_INFO: `/v3/projects/34ca99f1da904d608e324f65f6571250` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:53.777068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619257b-7aae-479a-9bf6-361ddf6dccb0 None admin] RBAC: Authorizing `identity:get_project(project_id=34ca99f1da904d608e324f65f6571250)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:53.778600 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619257b-7aae-479a-9bf6-361ddf6dccb0 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:53.780554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 196/399] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:53 2022] GET /identity/v3/projects/34ca99f1da904d608e324f65f6571250 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:53.781045 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31023445-59f4-470b-9784-c1fa13e0e5ad None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:53.781622 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167e016-7f88-4b74-a66c-76a3f67bdb8f None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:53.782076 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167e016-7f88-4b74-a66c-76a3f67bdb8f None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:53.782453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167e016-7f88-4b74-a66c-76a3f67bdb8f None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:53.791196 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 197/400] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:53.791196 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167e016-7f88-4b74-a66c-76a3f67bdb8f None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:53.822747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c831737-2b89-4b1c-9ad1-1b569c6e1585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:53.823484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c831737-2b89-4b1c-9ad1-1b569c6e1585 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:53.823781 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c831737-2b89-4b1c-9ad1-1b569c6e1585 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:53.824045 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c831737-2b89-4b1c-9ad1-1b569c6e1585 None admin] PATH_INFO: `/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317/groups/da3fbc722b5249c6b875074fee5f3ef5/roles/47cde553d7ee465bbe685a05c00b94e2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:53.826297 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c831737-2b89-4b1c-9ad1-1b569c6e1585 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a84e3fab8334ecca0cd4aae83cb5317, group_id=da3fbc722b5249c6b875074fee5f3ef5, role_id=47cde553d7ee465bbe685a05c00b94e2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:53.827729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c831737-2b89-4b1c-9ad1-1b569c6e1585 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:53.842301 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 204/401] 10.222.0.20 () {58 vars in 1442 bytes} [Sat Aug 6 12:17:53 2022] PUT /identity/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317/groups/da3fbc722b5249c6b875074fee5f3ef5/roles/47cde553d7ee465bbe685a05c00b94e2 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:17:53.843656 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221402d-c222-4eed-a9de-e5c8602c27d3 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:53.844138 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221402d-c222-4eed-a9de-e5c8602c27d3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:53.844439 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221402d-c222-4eed-a9de-e5c8602c27d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:53.880109 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3221402d-c222-4eed-a9de-e5c8602c27d3 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:53.881373 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3221402d-c222-4eed-a9de-e5c8602c27d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:53.964845 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167e016-7f88-4b74-a66c-76a3f67bdb8f None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:53.998353 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 205/402] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:54.019247 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc4bb46-d521-435b-ac80-1ab309808f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:54.019986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4bb46-d521-435b-ac80-1ab309808f3c None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:54.020340 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4bb46-d521-435b-ac80-1ab309808f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:54.020626 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4bb46-d521-435b-ac80-1ab309808f3c None admin] PATH_INFO: `/v3/projects/34ca99f1da904d608e324f65f6571250/groups/da3fbc722b5249c6b875074fee5f3ef5/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:54.022664 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc4bb46-d521-435b-ac80-1ab309808f3c None admin] RBAC: Authorizing `identity:create_grant(project_id=34ca99f1da904d608e324f65f6571250, group_id=da3fbc722b5249c6b875074fee5f3ef5, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:54.024257 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc4bb46-d521-435b-ac80-1ab309808f3c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:54.024381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221402d-c222-4eed-a9de-e5c8602c27d3 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:54.037305 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:54.048474 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 198/403] 10.222.0.20 () {58 vars in 1442 bytes} [Sat Aug 6 12:17:53 2022] PUT /identity/v3/projects/34ca99f1da904d608e324f65f6571250/groups/da3fbc722b5249c6b875074fee5f3ef5/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:17:54.119196 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39b741-da9e-4320-b2c3-1d0349b0929e None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:54.119551 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39b741-da9e-4320-b2c3-1d0349b0929e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:54.119805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39b741-da9e-4320-b2c3-1d0349b0929e None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:54.121551 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 199/404] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:17:54.127035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c7047-b368-41a3-9ead-182079e69dfd None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:54.127314 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c7047-b368-41a3-9ead-182079e69dfd None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:54.127583 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c7047-b368-41a3-9ead-182079e69dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:54.139676 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:54.151231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-9c3c7047-b368-41a3-9ead-182079e69dfd None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:54.152549 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9c3c7047-b368-41a3-9ead-182079e69dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:54.167395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221402d-c222-4eed-a9de-e5c8602c27d3 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:54.207628 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4669c7-3c49-4059-9ee9-bcecaff0fd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:54.208360 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4669c7-3c49-4059-9ee9-bcecaff0fd62 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:54.208651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4669c7-3c49-4059-9ee9-bcecaff0fd62 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:54.208925 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4669c7-3c49-4059-9ee9-bcecaff0fd62 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:54.220770 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4669c7-3c49-4059-9ee9-bcecaff0fd62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:54.225979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-6f4669c7-3c49-4059-9ee9-bcecaff0fd62 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:17:54.225979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:54.225979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:54.225979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:54.225979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:54.225979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:54.225979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:54.225979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:54.225979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:54.225979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:54.225979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:54.225979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:54.225979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:54.225979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:54.225979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:54.225979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:54.226904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:54.226904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:54.226904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:54.226904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:54.226904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:54.226904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:54.226904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:54.226904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:54.226904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:54.226904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:54.226904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:54.226904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:54.226904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:54.226904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:54.226904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:54.226904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:54.228830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:54.228830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:54.228830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:54.228830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:54.228830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:54.228830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:54.228830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:54.228830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:54.228830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:54.228830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:54.228830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:17:54.228830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:54.228830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 206/405] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Aug 6 12:17:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:17:54.233770 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4669c7-3c49-4059-9ee9-bcecaff0fd62 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:54.262967 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 200/406] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 12:17:54.286084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c7047-b368-41a3-9ead-182079e69dfd None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:54.346479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4669c7-3c49-4059-9ee9-bcecaff0fd62 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:54.382986 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e6fbe-4a31-4275-8e5c-cc63cd647b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:54.383609 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e6fbe-4a31-4275-8e5c-cc63cd647b61 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:54.383889 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e6fbe-4a31-4275-8e5c-cc63cd647b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:54.384166 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e6fbe-4a31-4275-8e5c-cc63cd647b61 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:54.384965 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e6fbe-4a31-4275-8e5c-cc63cd647b61 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:54.385267 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e6fbe-4a31-4275-8e5c-cc63cd647b61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:54.386651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e6fbe-4a31-4275-8e5c-cc63cd647b61 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:54.394526 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c7047-b368-41a3-9ead-182079e69dfd None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:54.395456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 207/407] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:17:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:54.401505 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e6fbe-4a31-4275-8e5c-cc63cd647b61 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:54.420692 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:54.445922 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:54.459265 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350b215-4b7a-4392-b80e-24e416fd6ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:54.459909 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350b215-4b7a-4392-b80e-24e416fd6ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:54.460192 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350b215-4b7a-4392-b80e-24e416fd6ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:54.460484 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350b215-4b7a-4392-b80e-24e416fd6ad8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:54.485630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350b215-4b7a-4392-b80e-24e416fd6ad8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:54.491130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-9350b215-4b7a-4392-b80e-24e416fd6ad8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:17:54.491130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:54.491130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:54.491130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:54.491130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:54.491130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:54.491130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:54.491130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:54.491130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:54.491130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:54.491130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:54.491130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:54.491130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:54.491130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:54.491130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:54.491130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:54.493224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:54.493224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:54.493224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:54.493224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:54.493224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:54.493224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:54.493224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:54.493224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:54.493224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:54.493224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:54.493224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:54.493224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:54.493224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:54.493224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:54.493224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:54.493224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:54.495190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:54.495190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:54.495190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:54.495190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:54.495190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:54.495190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:54.495190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:54.495190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:54.495190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:54.495190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:54.495190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:17:54.495190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:17:54.500819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 201/408] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:17:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:17:54.506651 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350b215-4b7a-4392-b80e-24e416fd6ad8 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:54.508828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e6fbe-4a31-4275-8e5c-cc63cd647b61 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:54.544733 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2fcc64-4492-4b25-a6de-e6c92e7522d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:54.545373 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2fcc64-4492-4b25-a6de-e6c92e7522d1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:54.545654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2fcc64-4492-4b25-a6de-e6c92e7522d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:54.545929 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2fcc64-4492-4b25-a6de-e6c92e7522d1 None admin] PATH_INFO: `/v3/groups/da3fbc722b5249c6b875074fee5f3ef5` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:54.547121 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2fcc64-4492-4b25-a6de-e6c92e7522d1 None admin] RBAC: Authorizing `identity:get_group(group_id=da3fbc722b5249c6b875074fee5f3ef5)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:54.548649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2fcc64-4492-4b25-a6de-e6c92e7522d1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:54.550935 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 208/409] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Aug 6 12:17:54 2022] GET /identity/v3/groups/da3fbc722b5249c6b875074fee5f3ef5 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:54.556906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2fcc64-4492-4b25-a6de-e6c92e7522d1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:54.638875 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:54.663702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2fcc64-4492-4b25-a6de-e6c92e7522d1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:54.701113 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd6494-e04e-4725-88a0-c0ea9c6c60cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:54.702195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd6494-e04e-4725-88a0-c0ea9c6c60cd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:54.702527 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd6494-e04e-4725-88a0-c0ea9c6c60cd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:54.702813 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd6494-e04e-4725-88a0-c0ea9c6c60cd None admin] PATH_INFO: `/v3/projects/34ca99f1da904d608e324f65f6571250` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:54.704822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd6494-e04e-4725-88a0-c0ea9c6c60cd None admin] RBAC: Authorizing `identity:get_project(project_id=34ca99f1da904d608e324f65f6571250)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:54.706619 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd6494-e04e-4725-88a0-c0ea9c6c60cd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:54.709573 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 209/410] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:54 2022] GET /identity/v3/projects/34ca99f1da904d608e324f65f6571250 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:54.715721 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd6494-e04e-4725-88a0-c0ea9c6c60cd None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:54.744143 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350b215-4b7a-4392-b80e-24e416fd6ad8 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:54.793225 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2a1bf-724a-4a07-ab87-5dc020952e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:54.794534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2a1bf-724a-4a07-ab87-5dc020952e6b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:54.795873 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2a1bf-724a-4a07-ab87-5dc020952e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:54.796420 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2a1bf-724a-4a07-ab87-5dc020952e6b None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:54.797402 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2a1bf-724a-4a07-ab87-5dc020952e6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:54.797940 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2a1bf-724a-4a07-ab87-5dc020952e6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:54.799517 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2a1bf-724a-4a07-ab87-5dc020952e6b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:54.812433 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 202/411] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:54.814596 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f5e99-d164-4f08-8bd1-668a94179ebf None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:54.815094 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f5e99-d164-4f08-8bd1-668a94179ebf None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:54.815870 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f5e99-d164-4f08-8bd1-668a94179ebf None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:54.822692 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 203/412] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:54.822692 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f5e99-d164-4f08-8bd1-668a94179ebf None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:54.835114 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd6494-e04e-4725-88a0-c0ea9c6c60cd None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:54.876731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:54.885884 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91fd5c5-ef1a-45ee-94e5-c6fe6d04620d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:54.887155 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fd5c5-ef1a-45ee-94e5-c6fe6d04620d None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:54.888055 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fd5c5-ef1a-45ee-94e5-c6fe6d04620d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:54.888841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fd5c5-ef1a-45ee-94e5-c6fe6d04620d None admin] PATH_INFO: `/v3/projects/34ca99f1da904d608e324f65f6571250/groups/da3fbc722b5249c6b875074fee5f3ef5/roles/47cde553d7ee465bbe685a05c00b94e2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:54.892256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91fd5c5-ef1a-45ee-94e5-c6fe6d04620d None admin] RBAC: Authorizing `identity:create_grant(project_id=34ca99f1da904d608e324f65f6571250, group_id=da3fbc722b5249c6b875074fee5f3ef5, role_id=47cde553d7ee465bbe685a05c00b94e2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:54.893976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91fd5c5-ef1a-45ee-94e5-c6fe6d04620d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:54.914401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 210/413] 10.222.0.20 () {58 vars in 1442 bytes} [Sat Aug 6 12:17:54 2022] PUT /identity/v3/projects/34ca99f1da904d608e324f65f6571250/groups/da3fbc722b5249c6b875074fee5f3ef5/roles/47cde553d7ee465bbe685a05c00b94e2 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:17:54.916959 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b50d6-ea00-4c17-8b5d-07df86383213 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:54.917488 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b50d6-ea00-4c17-8b5d-07df86383213 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:54.918004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b50d6-ea00-4c17-8b5d-07df86383213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:54.927263 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:54.946071 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a19b50d6-ea00-4c17-8b5d-07df86383213 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:54.949341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a19b50d6-ea00-4c17-8b5d-07df86383213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:54.972001 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f5e99-d164-4f08-8bd1-668a94179ebf None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:55.036752 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 211/414] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:55.067481 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c0a20c-1bf7-4ee8-897c-302fd2af1ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:55.068146 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0a20c-1bf7-4ee8-897c-302fd2af1ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:55.068452 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0a20c-1bf7-4ee8-897c-302fd2af1ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:55.068733 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0a20c-1bf7-4ee8-897c-302fd2af1ca4 None admin] PATH_INFO: `/v3/groups/d78e995107bc436bb56514be0b1f1a06` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:55.072686 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b50d6-ea00-4c17-8b5d-07df86383213 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:55.073173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c0a20c-1bf7-4ee8-897c-302fd2af1ca4 None admin] RBAC: Authorizing `identity:get_group(group_id=d78e995107bc436bb56514be0b1f1a06)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:55.074703 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c0a20c-1bf7-4ee8-897c-302fd2af1ca4 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:55.076398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 204/415] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Aug 6 12:17:54 2022] GET /identity/v3/groups/d78e995107bc436bb56514be0b1f1a06 => generated 229 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:55.081843 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c0a20c-1bf7-4ee8-897c-302fd2af1ca4 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:55.199923 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b50d6-ea00-4c17-8b5d-07df86383213 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:55.239569 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995ac9a-07c6-4155-a5d5-2c15f9f47c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:55.240249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995ac9a-07c6-4155-a5d5-2c15f9f47c47 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:55.240561 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995ac9a-07c6-4155-a5d5-2c15f9f47c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:55.240835 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995ac9a-07c6-4155-a5d5-2c15f9f47c47 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:55.243214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:55.255706 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b995ac9a-07c6-4155-a5d5-2c15f9f47c47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:55.261192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-b995ac9a-07c6-4155-a5d5-2c15f9f47c47 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:55.261192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:55.261192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:55.261192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:55.261192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:55.261192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:55.261192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:55.261192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:55.261192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:55.261192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:55.261192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:55.261192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:55.261192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:55.261192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:55.261192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:55.261192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:55.262084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:55.262084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:55.262084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:55.262084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:55.262084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:55.262084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:55.262084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:55.262084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:55.262084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:55.262084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:55.262084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:55.262084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:55.262084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:55.262084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:55.262084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:55.262084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:55.263276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:55.263276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:55.263276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:55.263276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:55.263276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:55.263276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:55.263276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:55.263276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:55.263276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:55.263276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:55.263276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:55.263276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:55.263276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 212/416] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:55.266285 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c0a20c-1bf7-4ee8-897c-302fd2af1ca4 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:55.268598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995ac9a-07c6-4155-a5d5-2c15f9f47c47 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:55.293326 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:55.315028 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa5c89-865b-404e-9ce2-df0acac3e472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:55.315813 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa5c89-865b-404e-9ce2-df0acac3e472 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:55.316167 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa5c89-865b-404e-9ce2-df0acac3e472 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:55.316548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa5c89-865b-404e-9ce2-df0acac3e472 None admin] PATH_INFO: `/v3/projects/7ad154ab2bd64d7e8e712fcec9e1c94f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:55.317834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa5c89-865b-404e-9ce2-df0acac3e472 None admin] RBAC: Authorizing `identity:get_project(project_id=7ad154ab2bd64d7e8e712fcec9e1c94f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:55.319383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa5c89-865b-404e-9ce2-df0acac3e472 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:55.322258 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 205/417] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:55 2022] GET /identity/v3/projects/7ad154ab2bd64d7e8e712fcec9e1c94f => generated 344 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:55.327607 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa5c89-865b-404e-9ce2-df0acac3e472 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:55.418850 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995ac9a-07c6-4155-a5d5-2c15f9f47c47 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:55.451965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa5c89-865b-404e-9ce2-df0acac3e472 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:55.492032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9961cb-6737-40ab-9667-128bd81e5586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:55.492881 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9961cb-6737-40ab-9667-128bd81e5586 None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:55.493271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9961cb-6737-40ab-9667-128bd81e5586 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:55.493651 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9961cb-6737-40ab-9667-128bd81e5586 None admin] PATH_INFO: `/v3/projects/7ad154ab2bd64d7e8e712fcec9e1c94f/groups/d78e995107bc436bb56514be0b1f1a06/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:55.496269 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9961cb-6737-40ab-9667-128bd81e5586 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ad154ab2bd64d7e8e712fcec9e1c94f, group_id=d78e995107bc436bb56514be0b1f1a06, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:55.497847 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9961cb-6737-40ab-9667-128bd81e5586 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:55.499189 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd43d3-462d-4d1b-a387-258179e0fe27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:55.499973 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd43d3-462d-4d1b-a387-258179e0fe27 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:55.500506 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd43d3-462d-4d1b-a387-258179e0fe27 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:55.500935 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd43d3-462d-4d1b-a387-258179e0fe27 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:55.501903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd43d3-462d-4d1b-a387-258179e0fe27 None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:55.502567 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd43d3-462d-4d1b-a387-258179e0fe27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:55.504143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd43d3-462d-4d1b-a387-258179e0fe27 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:55.513337 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 206/418] 10.222.0.20 () {58 vars in 1442 bytes} [Sat Aug 6 12:17:55 2022] PUT /identity/v3/projects/7ad154ab2bd64d7e8e712fcec9e1c94f/groups/d78e995107bc436bb56514be0b1f1a06/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:17:55.535056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 213/419] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:55.541472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9961cb-6737-40ab-9667-128bd81e5586 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:55.596436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:55.659364 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9961cb-6737-40ab-9667-128bd81e5586 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:55.696997 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc9c56-8d9d-41f0-9475-8ceb8a2a3cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:55.698062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc9c56-8d9d-41f0-9475-8ceb8a2a3cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:55.698409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc9c56-8d9d-41f0-9475-8ceb8a2a3cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:55.698692 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc9c56-8d9d-41f0-9475-8ceb8a2a3cc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:55.771444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc9c56-8d9d-41f0-9475-8ceb8a2a3cc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:55.772892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc9c56-8d9d-41f0-9475-8ceb8a2a3cc5 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:55.820314 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:55.881627 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 207/420] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:55.887505 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd43d3-462d-4d1b-a387-258179e0fe27 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:55.984087 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:56.057441 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd43d3-462d-4d1b-a387-258179e0fe27 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:56.104973 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d348328-7409-4ed8-b033-16a4512c9ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:56.105610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d348328-7409-4ed8-b033-16a4512c9ebf None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:56.105892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d348328-7409-4ed8-b033-16a4512c9ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:56.106169 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d348328-7409-4ed8-b033-16a4512c9ebf None admin] PATH_INFO: `/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:56.108367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d348328-7409-4ed8-b033-16a4512c9ebf None admin] RBAC: Authorizing `identity:get_project(project_id=5a84e3fab8334ecca0cd4aae83cb5317)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:56.109815 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d348328-7409-4ed8-b033-16a4512c9ebf None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:56.111642 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 214/421] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:55 2022] GET /identity/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:56.124179 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc9c56-8d9d-41f0-9475-8ceb8a2a3cc5 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:56.260108 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc9c56-8d9d-41f0-9475-8ceb8a2a3cc5 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:56.305123 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca2891-0b81-42b3-a198-9c39d8c6381a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:56.305733 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca2891-0b81-42b3-a198-9c39d8c6381a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:56.306015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca2891-0b81-42b3-a198-9c39d8c6381a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:56.306324 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca2891-0b81-42b3-a198-9c39d8c6381a None admin] PATH_INFO: `/v3/groups/da3fbc722b5249c6b875074fee5f3ef5` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:56.307558 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca2891-0b81-42b3-a198-9c39d8c6381a None admin] RBAC: Authorizing `identity:get_group(group_id=da3fbc722b5249c6b875074fee5f3ef5)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:56.308989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca2891-0b81-42b3-a198-9c39d8c6381a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:56.310885 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 208/422] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Aug 6 12:17:56 2022] GET /identity/v3/groups/da3fbc722b5249c6b875074fee5f3ef5 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:56.313790 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14c268-b77c-44c1-b900-a04a41496cc5 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:56.314329 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14c268-b77c-44c1-b900-a04a41496cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:56.314797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14c268-b77c-44c1-b900-a04a41496cc5 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:56.316251 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca2891-0b81-42b3-a198-9c39d8c6381a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:56.317039 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 215/423] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:56.322943 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7994d84-d2b2-414b-a8c5-54f466a9a62b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:56.323487 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7994d84-d2b2-414b-a8c5-54f466a9a62b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:56.323990 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7994d84-d2b2-414b-a8c5-54f466a9a62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:56.342387 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e7994d84-d2b2-414b-a8c5-54f466a9a62b None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:56.343832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e7994d84-d2b2-414b-a8c5-54f466a9a62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:56.356660 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:56.458357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 216/424] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 12:17:56.464243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca2891-0b81-42b3-a198-9c39d8c6381a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:56.493167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7994d84-d2b2-414b-a8c5-54f466a9a62b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:56.538837 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da974d-b5e4-453a-bae1-b324b34b4316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:56.540415 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da974d-b5e4-453a-bae1-b324b34b4316 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:56.541018 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da974d-b5e4-453a-bae1-b324b34b4316 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:56.542081 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da974d-b5e4-453a-bae1-b324b34b4316 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:56.543184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da974d-b5e4-453a-bae1-b324b34b4316 None admin] RBAC: Adding query filter params (group.id=da3fbc722b5249c6b875074fee5f3ef5, role.id=0fd7b75299b64c54971d9bfeaec30f4e, scope.project.id=5a84e3fab8334ecca0cd4aae83cb5317) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:56.544785 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da974d-b5e4-453a-bae1-b324b34b4316 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:56.547273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da974d-b5e4-453a-bae1-b324b34b4316 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:56.561744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 209/425] 10.222.0.20 () {56 vars in 1376 bytes} [Sat Aug 6 12:17:56 2022] GET /identity/v3/role_assignments?group.id=da3fbc722b5249c6b875074fee5f3ef5&scope.project.id=5a84e3fab8334ecca0cd4aae83cb5317&role.id=0fd7b75299b64c54971d9bfeaec30f4e => generated 611 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:56.580095 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:56.631581 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7994d84-d2b2-414b-a8c5-54f466a9a62b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:56.682144 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585fc55-ae16-4acb-90c8-c379013f9d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:56.682816 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c010580-4b7c-4888-a2d2-f558d384dbdd None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:56.683213 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c010580-4b7c-4888-a2d2-f558d384dbdd None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:56.683369 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585fc55-ae16-4acb-90c8-c379013f9d3a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:56.683675 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c010580-4b7c-4888-a2d2-f558d384dbdd None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:56.683918 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585fc55-ae16-4acb-90c8-c379013f9d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:56.685581 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 210/426] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:17:56.685761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585fc55-ae16-4acb-90c8-c379013f9d3a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:56.691268 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ca814-bb43-4b5c-9dab-059c22023b7d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:56.691646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ca814-bb43-4b5c-9dab-059c22023b7d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:56.692043 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ca814-bb43-4b5c-9dab-059c22023b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:56.705901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585fc55-ae16-4acb-90c8-c379013f9d3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:56.711061 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-f69ca814-bb43-4b5c-9dab-059c22023b7d None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:56.711610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-f585fc55-ae16-4acb-90c8-c379013f9d3a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:17:56.711610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:56.711610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:56.711610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:56.711610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:56.711610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:56.711610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:56.711610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:56.711610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:56.711610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:56.711610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:56.711610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:56.711610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:56.711610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:56.711610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:56.711610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:56.713238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:56.713238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:56.713238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:56.713238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:56.713238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:56.713238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:56.713238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:56.713238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:56.713238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:56.713238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:56.713238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:56.713238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:56.713238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:56.713238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:56.713238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:56.713238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:56.714249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:56.714249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:56.714249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:56.714249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:56.714249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:56.714249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:56.714249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:56.714249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:56.714249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:56.714249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:56.714249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:17:56.714249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:56.715504 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f69ca814-bb43-4b5c-9dab-059c22023b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:56.716783 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 217/427] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Aug 6 12:17:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:17:56.722423 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585fc55-ae16-4acb-90c8-c379013f9d3a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:56.841742 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 211/428] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 12:17:56.846633 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585fc55-ae16-4acb-90c8-c379013f9d3a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:56.861956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:56.870369 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ca814-bb43-4b5c-9dab-059c22023b7d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:56.887381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:56.893420 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385604f9-96ed-4526-b077-af525be05a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:56.894094 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385604f9-96ed-4526-b077-af525be05a6d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:56.894341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385604f9-96ed-4526-b077-af525be05a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:56.894827 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385604f9-96ed-4526-b077-af525be05a6d None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:56.896053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385604f9-96ed-4526-b077-af525be05a6d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:56.896437 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385604f9-96ed-4526-b077-af525be05a6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:56.898144 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385604f9-96ed-4526-b077-af525be05a6d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:56.909314 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 218/429] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:17:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:56.914963 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385604f9-96ed-4526-b077-af525be05a6d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:57.021018 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ca814-bb43-4b5c-9dab-059c22023b7d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:57.059269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385604f9-96ed-4526-b077-af525be05a6d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:57.060978 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec358c-a21b-4003-8d89-0dd319616654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:57.061712 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec358c-a21b-4003-8d89-0dd319616654 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:57.062089 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec358c-a21b-4003-8d89-0dd319616654 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:57.062469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec358c-a21b-4003-8d89-0dd319616654 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:57.083371 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec358c-a21b-4003-8d89-0dd319616654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:57.089842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-8fec358c-a21b-4003-8d89-0dd319616654 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:57.089842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:57.089842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:57.089842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:57.089842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:57.089842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:57.089842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:57.089842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:57.089842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:57.089842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:57.089842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:57.089842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:57.089842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:57.089842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:57.089842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:57.089842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:57.090776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:57.090776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:57.090776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:57.090776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:57.090776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:57.090776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:57.090776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:57.090776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:57.090776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:57.090776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:57.090776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:57.090776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:57.090776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:57.090776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:57.090776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:57.090776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:57.091568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:57.091568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:57.091568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:57.091568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:57.091568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:57.091568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:57.091568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:57.091568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:57.091568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:57.091568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:57.091568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 06 12:17:57.091568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:17:57.092209 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 212/430] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:57.098179 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec358c-a21b-4003-8d89-0dd319616654 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:57.115081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701bde25-073e-4d35-ba04-e51a9be62980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:57.116389 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701bde25-073e-4d35-ba04-e51a9be62980 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:57.116990 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701bde25-073e-4d35-ba04-e51a9be62980 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:57.117461 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701bde25-073e-4d35-ba04-e51a9be62980 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:57.174626 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:57.191952 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:57.216263 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701bde25-073e-4d35-ba04-e51a9be62980 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:57.222219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701bde25-073e-4d35-ba04-e51a9be62980 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:57.245689 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec358c-a21b-4003-8d89-0dd319616654 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:57.293240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5329fe-b3a8-4a9b-be8c-a97b689ff261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:57.293865 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5329fe-b3a8-4a9b-be8c-a97b689ff261 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:57.294147 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5329fe-b3a8-4a9b-be8c-a97b689ff261 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:57.294425 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5329fe-b3a8-4a9b-be8c-a97b689ff261 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:57.295674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5329fe-b3a8-4a9b-be8c-a97b689ff261 None admin] RBAC: Adding query filter params (name=member) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:57.295986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5329fe-b3a8-4a9b-be8c-a97b689ff261 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:57.297993 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5329fe-b3a8-4a9b-be8c-a97b689ff261 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:57.316358 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 213/431] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:17:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:57.316358 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5329fe-b3a8-4a9b-be8c-a97b689ff261 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:57.356734 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 219/432] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:57.362484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701bde25-073e-4d35-ba04-e51a9be62980 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:57.429651 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5329fe-b3a8-4a9b-be8c-a97b689ff261 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:57.443838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:57.464110 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:57.471454 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701bde25-073e-4d35-ba04-e51a9be62980 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:57.471824 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf329a2d-8492-429e-a25e-45ff52ab3dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:57.472702 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf329a2d-8492-429e-a25e-45ff52ab3dca None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:57.473238 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf329a2d-8492-429e-a25e-45ff52ab3dca None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:57.473800 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf329a2d-8492-429e-a25e-45ff52ab3dca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:57.508412 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c496a-bc1e-4f8b-beec-53ce8c89f8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:57.509017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c496a-bc1e-4f8b-beec-53ce8c89f8ee None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:57.509325 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c496a-bc1e-4f8b-beec-53ce8c89f8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:57.509643 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c496a-bc1e-4f8b-beec-53ce8c89f8ee None admin] PATH_INFO: `/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:57.510835 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c496a-bc1e-4f8b-beec-53ce8c89f8ee None admin] RBAC: Authorizing `identity:get_project(project_id=5a84e3fab8334ecca0cd4aae83cb5317)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:57.512424 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c496a-bc1e-4f8b-beec-53ce8c89f8ee None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:57.518908 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 220/433] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:57 2022] GET /identity/v3/projects/5a84e3fab8334ecca0cd4aae83cb5317 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:57.520390 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c496a-bc1e-4f8b-beec-53ce8c89f8ee None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:57.548867 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf329a2d-8492-429e-a25e-45ff52ab3dca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:57.550876 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf329a2d-8492-429e-a25e-45ff52ab3dca None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:57.632898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c496a-bc1e-4f8b-beec-53ce8c89f8ee None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:57.667705 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e4935-78ec-453b-b6a9-246061f2034d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:57.668366 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e4935-78ec-453b-b6a9-246061f2034d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:57.668650 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e4935-78ec-453b-b6a9-246061f2034d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:57.668922 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e4935-78ec-453b-b6a9-246061f2034d None admin] PATH_INFO: `/v3/groups/da3fbc722b5249c6b875074fee5f3ef5` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:57.673668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 214/434] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:57.673792 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e4935-78ec-453b-b6a9-246061f2034d None admin] RBAC: Authorizing `identity:get_group(group_id=da3fbc722b5249c6b875074fee5f3ef5)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:57.673867 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba09eb9-9411-4fb7-a842-b950c571976c None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:57.673867 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba09eb9-9411-4fb7-a842-b950c571976c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:57.674005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e4935-78ec-453b-b6a9-246061f2034d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:57.674077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba09eb9-9411-4fb7-a842-b950c571976c None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:57.674077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 215/435] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:57.674709 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba09eb9-9411-4fb7-a842-b950c571976c None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:57.676750 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 221/436] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Aug 6 12:17:57 2022] GET /identity/v3/groups/da3fbc722b5249c6b875074fee5f3ef5 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:57.678170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed11887-d0f8-4f37-bd4a-fd67551d4776 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:57.678456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed11887-d0f8-4f37-bd4a-fd67551d4776 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:57.678727 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed11887-d0f8-4f37-bd4a-fd67551d4776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:57.702055 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3ed11887-d0f8-4f37-bd4a-fd67551d4776 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:57.703657 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3ed11887-d0f8-4f37-bd4a-fd67551d4776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:57.711242 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:57.712596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:57.780705 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 222/437] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:57.781925 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed11887-d0f8-4f37-bd4a-fd67551d4776 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:57.787982 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba09eb9-9411-4fb7-a842-b950c571976c None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:57.827799 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278afe3-002f-4bfa-9b89-88dda78c470f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:57.828765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278afe3-002f-4bfa-9b89-88dda78c470f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:57.829403 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278afe3-002f-4bfa-9b89-88dda78c470f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:57.829682 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278afe3-002f-4bfa-9b89-88dda78c470f None admin] PATH_INFO: `/v3/projects/34ca99f1da904d608e324f65f6571250` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:57.830855 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278afe3-002f-4bfa-9b89-88dda78c470f None admin] RBAC: Authorizing `identity:get_project(project_id=34ca99f1da904d608e324f65f6571250)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:57.832513 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278afe3-002f-4bfa-9b89-88dda78c470f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:57.838742 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 216/438] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:57 2022] GET /identity/v3/projects/34ca99f1da904d608e324f65f6571250 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:57.838742 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278afe3-002f-4bfa-9b89-88dda78c470f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:57.894220 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed11887-d0f8-4f37-bd4a-fd67551d4776 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:57.935452 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dab3e6-30ab-4692-b747-26660be0c308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:57.936315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dab3e6-30ab-4692-b747-26660be0c308 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:57.936599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dab3e6-30ab-4692-b747-26660be0c308 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:57.936869 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dab3e6-30ab-4692-b747-26660be0c308 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:57.937672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dab3e6-30ab-4692-b747-26660be0c308 None admin] RBAC: Adding query filter params (group.id=da3fbc722b5249c6b875074fee5f3ef5, role.id=47cde553d7ee465bbe685a05c00b94e2, scope.project.id=5a84e3fab8334ecca0cd4aae83cb5317) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:57.938059 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dab3e6-30ab-4692-b747-26660be0c308 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:57.939648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dab3e6-30ab-4692-b747-26660be0c308 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:57.940984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:57.945027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278afe3-002f-4bfa-9b89-88dda78c470f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:57.947942 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 223/439] 10.222.0.20 () {56 vars in 1376 bytes} [Sat Aug 6 12:17:57 2022] GET /identity/v3/role_assignments?group.id=da3fbc722b5249c6b875074fee5f3ef5&scope.project.id=5a84e3fab8334ecca0cd4aae83cb5317&role.id=47cde553d7ee465bbe685a05c00b94e2 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:57.948775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dab3e6-30ab-4692-b747-26660be0c308 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:57.957909 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:58.004165 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa55fb-dfc8-4726-b2bc-fe080e7b4d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:58.004864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa55fb-dfc8-4726-b2bc-fe080e7b4d3b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:58.005180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa55fb-dfc8-4726-b2bc-fe080e7b4d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:58.005450 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa55fb-dfc8-4726-b2bc-fe080e7b4d3b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:58.018243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa55fb-dfc8-4726-b2bc-fe080e7b4d3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:58.023766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-dfaa55fb-dfc8-4726-b2bc-fe080e7b4d3b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:17:58.023766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:58.023766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:58.023766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:58.023766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:58.023766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:58.023766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:58.023766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:58.023766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:58.023766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:58.023766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:58.023766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:58.023766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:58.023766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:58.023766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:58.023766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:58.024754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:58.024754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:58.024754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:58.024754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:58.024754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:58.024754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:58.024754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:58.024754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:58.024754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:58.024754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:58.024754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:58.024754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:58.024754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:58.024754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:58.024754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:58.024754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:58.025605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:58.025605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:58.025605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:58.025605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:58.025605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:58.025605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:58.025605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:58.025605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:58.025605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:58.025605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:58.025605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:17:58.025605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:17:58.030064 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 217/440] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Aug 6 12:17:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:58.031277 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa55fb-dfc8-4726-b2bc-fe080e7b4d3b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:58.086919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dab3e6-30ab-4692-b747-26660be0c308 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:58.134451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784e88e-88fe-4a38-acc3-531cf7955ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:58.135587 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784e88e-88fe-4a38-acc3-531cf7955ecf None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:58.135587 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784e88e-88fe-4a38-acc3-531cf7955ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:58.135754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784e88e-88fe-4a38-acc3-531cf7955ecf None admin] PATH_INFO: `/v3/groups/da3fbc722b5249c6b875074fee5f3ef5` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:58.138658 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b784e88e-88fe-4a38-acc3-531cf7955ecf None admin] RBAC: Authorizing `identity:get_group(group_id=da3fbc722b5249c6b875074fee5f3ef5)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:58.138658 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b784e88e-88fe-4a38-acc3-531cf7955ecf None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:58.140039 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 224/441] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Aug 6 12:17:57 2022] GET /identity/v3/groups/da3fbc722b5249c6b875074fee5f3ef5 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:58.141024 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f39fb-5e4b-420d-9861-48074e99413e None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:58.142809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f39fb-5e4b-420d-9861-48074e99413e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:58.142809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f39fb-5e4b-420d-9861-48074e99413e None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:58.142809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 225/442] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:17:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:17:58.147478 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f39fb-5e4b-420d-9861-48074e99413e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:58.148584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa55fb-dfc8-4726-b2bc-fe080e7b4d3b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:58.189863 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e4f8dd-466c-4bb0-a469-2be9a51117d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:58.190955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4f8dd-466c-4bb0-a469-2be9a51117d1 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:58.190955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4f8dd-466c-4bb0-a469-2be9a51117d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:58.191128 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4f8dd-466c-4bb0-a469-2be9a51117d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:58.191707 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e4f8dd-466c-4bb0-a469-2be9a51117d1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:58.192008 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e4f8dd-466c-4bb0-a469-2be9a51117d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:58.193424 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e4f8dd-466c-4bb0-a469-2be9a51117d1 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:58.202866 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 218/443] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:17:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:58.204199 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aface7-5dc3-4bdb-9d08-d31bc153f1e1 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:58.204437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aface7-5dc3-4bdb-9d08-d31bc153f1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:58.204714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aface7-5dc3-4bdb-9d08-d31bc153f1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:58.224405 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c5aface7-5dc3-4bdb-9d08-d31bc153f1e1 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:17:58.225597 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c5aface7-5dc3-4bdb-9d08-d31bc153f1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:17:58.237292 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:58.242402 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:58.274061 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f39fb-5e4b-420d-9861-48074e99413e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:58.306062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 219/444] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:17:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:17:58.306977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aface7-5dc3-4bdb-9d08-d31bc153f1e1 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:58.323429 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4edd9-36a8-4d49-a533-a2c34353c9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:58.324622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4edd9-36a8-4d49-a533-a2c34353c9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:58.324910 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4edd9-36a8-4d49-a533-a2c34353c9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:58.326101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4edd9-36a8-4d49-a533-a2c34353c9c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:58.326101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4edd9-36a8-4d49-a533-a2c34353c9c3 None admin] RBAC: Adding query filter params (group.id=da3fbc722b5249c6b875074fee5f3ef5, role.id=0fd7b75299b64c54971d9bfeaec30f4e, scope.project.id=34ca99f1da904d608e324f65f6571250) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:58.326268 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4edd9-36a8-4d49-a533-a2c34353c9c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:58.328203 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4edd9-36a8-4d49-a533-a2c34353c9c3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:58.338096 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 226/445] 10.222.0.20 () {56 vars in 1376 bytes} [Sat Aug 6 12:17:58 2022] GET /identity/v3/role_assignments?group.id=da3fbc722b5249c6b875074fee5f3ef5&scope.project.id=34ca99f1da904d608e324f65f6571250&role.id=0fd7b75299b64c54971d9bfeaec30f4e => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:58.338977 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4edd9-36a8-4d49-a533-a2c34353c9c3 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:58.424376 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aface7-5dc3-4bdb-9d08-d31bc153f1e1 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:58.460421 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4edd9-36a8-4d49-a533-a2c34353c9c3 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:58.461911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abb34a-0903-45ff-a37b-fb74fb06e005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:58.463154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abb34a-0903-45ff-a37b-fb74fb06e005 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:58.463507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abb34a-0903-45ff-a37b-fb74fb06e005 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:58.463864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abb34a-0903-45ff-a37b-fb74fb06e005 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:58.484749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:58.500727 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ee862-c46f-493a-a5b8-51de1ba135a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:58.501367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ee862-c46f-493a-a5b8-51de1ba135a2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:58.501643 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ee862-c46f-493a-a5b8-51de1ba135a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:58.501914 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ee862-c46f-493a-a5b8-51de1ba135a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:58.510885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:58.516676 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ee862-c46f-493a-a5b8-51de1ba135a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:17:58.521622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-015ee862-c46f-493a-a5b8-51de1ba135a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:17:58.521622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:17:58.521622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:17:58.521622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:17:58.521622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:17:58.521622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:17:58.521622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:17:58.521622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:17:58.521622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:17:58.521622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:17:58.521622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:17:58.521622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:17:58.521622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:17:58.521622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:17:58.521622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:17:58.521622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:17:58.522479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:17:58.522479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:17:58.522479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:17:58.522479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:17:58.522479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:17:58.522479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:17:58.522479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:17:58.522479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:17:58.522479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:17:58.522479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:17:58.522479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:17:58.522479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:17:58.522479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:17:58.522479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:17:58.522479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:17:58.522479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:17:58.523258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:17:58.523258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:17:58.523258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:17:58.523258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:17:58.523258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:17:58.523258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:17:58.523258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:17:58.523258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:17:58.523258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:17:58.523258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:17:58.523258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:17:58.523258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:17:58.523258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 227/446] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:17:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:17:58.528746 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ee862-c46f-493a-a5b8-51de1ba135a2 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:58.549828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abb34a-0903-45ff-a37b-fb74fb06e005 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:58.551275 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abb34a-0903-45ff-a37b-fb74fb06e005 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:58.654785 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ee862-c46f-493a-a5b8-51de1ba135a2 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:58.682768 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 220/447] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:17:58.688055 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abb34a-0903-45ff-a37b-fb74fb06e005 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:58.694127 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b69c9-a363-4ece-acd5-0b4b2417048b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:58.694756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b69c9-a363-4ece-acd5-0b4b2417048b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:58.695039 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b69c9-a363-4ece-acd5-0b4b2417048b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:58.695321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b69c9-a363-4ece-acd5-0b4b2417048b None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:58.696043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b69c9-a363-4ece-acd5-0b4b2417048b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:58.696381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b69c9-a363-4ece-acd5-0b4b2417048b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:58.698316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b69c9-a363-4ece-acd5-0b4b2417048b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:58.706940 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 228/448] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:17:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:58.712796 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b69c9-a363-4ece-acd5-0b4b2417048b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:58.738926 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:58.771358 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:58.788079 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abb34a-0903-45ff-a37b-fb74fb06e005 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:58.811424 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b69c9-a363-4ece-acd5-0b4b2417048b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:58.822101 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfd7ab-7834-4f89-bad0-603f2e14e87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:58.822723 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfd7ab-7834-4f89-bad0-603f2e14e87c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:58.822997 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfd7ab-7834-4f89-bad0-603f2e14e87c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:58.823264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfd7ab-7834-4f89-bad0-603f2e14e87c None admin] PATH_INFO: `/v3/projects/34ca99f1da904d608e324f65f6571250` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:58.824431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfd7ab-7834-4f89-bad0-603f2e14e87c None admin] RBAC: Authorizing `identity:get_project(project_id=34ca99f1da904d608e324f65f6571250)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:58.825860 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfd7ab-7834-4f89-bad0-603f2e14e87c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:58.827480 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 221/449] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:58 2022] GET /identity/v3/projects/34ca99f1da904d608e324f65f6571250 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:58.832840 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfd7ab-7834-4f89-bad0-603f2e14e87c None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:58.843365 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49dd07-f0c7-4ce0-b513-256845ef35ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:58.843970 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49dd07-f0c7-4ce0-b513-256845ef35ec None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:58.844246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49dd07-f0c7-4ce0-b513-256845ef35ec None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:58.844545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49dd07-f0c7-4ce0-b513-256845ef35ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:58.909393 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49dd07-f0c7-4ce0-b513-256845ef35ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:58.910817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49dd07-f0c7-4ce0-b513-256845ef35ec None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:58.929054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfd7ab-7834-4f89-bad0-603f2e14e87c None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:58.946294 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:58.965437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba31c5-6dff-4544-9c87-d16cc4778157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:58.966068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba31c5-6dff-4544-9c87-d16cc4778157 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:58.966349 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba31c5-6dff-4544-9c87-d16cc4778157 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:58.966625 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba31c5-6dff-4544-9c87-d16cc4778157 None admin] PATH_INFO: `/v3/groups/da3fbc722b5249c6b875074fee5f3ef5` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:58.967277 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:58.967776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba31c5-6dff-4544-9c87-d16cc4778157 None admin] RBAC: Authorizing `identity:get_group(group_id=da3fbc722b5249c6b875074fee5f3ef5)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:58.969259 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba31c5-6dff-4544-9c87-d16cc4778157 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:58.970842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 222/450] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Aug 6 12:17:58 2022] GET /identity/v3/groups/da3fbc722b5249c6b875074fee5f3ef5 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:58.976479 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba31c5-6dff-4544-9c87-d16cc4778157 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:59.010445 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 229/451] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:17:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 06 12:17:59.015980 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49dd07-f0c7-4ce0-b513-256845ef35ec None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:59.076674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba31c5-6dff-4544-9c87-d16cc4778157 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:59.111469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dcfc0f-f4f7-49f8-925b-9f119eb58105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:59.111669 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49dd07-f0c7-4ce0-b513-256845ef35ec None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:59.112137 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dcfc0f-f4f7-49f8-925b-9f119eb58105 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:59.112449 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dcfc0f-f4f7-49f8-925b-9f119eb58105 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:59.112731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dcfc0f-f4f7-49f8-925b-9f119eb58105 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:59.113519 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dcfc0f-f4f7-49f8-925b-9f119eb58105 None admin] RBAC: Adding query filter params (group.id=da3fbc722b5249c6b875074fee5f3ef5, role.id=47cde553d7ee465bbe685a05c00b94e2, scope.project.id=34ca99f1da904d608e324f65f6571250) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:59.113863 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dcfc0f-f4f7-49f8-925b-9f119eb58105 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:59.115299 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dcfc0f-f4f7-49f8-925b-9f119eb58105 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:59.121860 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 223/452] 10.222.0.20 () {56 vars in 1376 bytes} [Sat Aug 6 12:17:58 2022] GET /identity/v3/role_assignments?group.id=da3fbc722b5249c6b875074fee5f3ef5&scope.project.id=34ca99f1da904d608e324f65f6571250&role.id=47cde553d7ee465bbe685a05c00b94e2 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:59.146024 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32565d-c352-477f-a4c0-eb7e3952e070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:59.146683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32565d-c352-477f-a4c0-eb7e3952e070 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:59.146962 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32565d-c352-477f-a4c0-eb7e3952e070 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:59.147233 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32565d-c352-477f-a4c0-eb7e3952e070 None admin] PATH_INFO: `/v3/projects/7ad154ab2bd64d7e8e712fcec9e1c94f` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:59.148507 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32565d-c352-477f-a4c0-eb7e3952e070 None admin] RBAC: Authorizing `identity:get_project(project_id=7ad154ab2bd64d7e8e712fcec9e1c94f)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:59.150520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32565d-c352-477f-a4c0-eb7e3952e070 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:59.152242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 230/453] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:17:59 2022] GET /identity/v3/projects/7ad154ab2bd64d7e8e712fcec9e1c94f => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:17:59.157997 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dcfc0f-f4f7-49f8-925b-9f119eb58105 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:59.210245 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:59.263679 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dcfc0f-f4f7-49f8-925b-9f119eb58105 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:59.298372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf5891a-3ce7-40b6-adcb-2efc26043fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:59.299014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5891a-3ce7-40b6-adcb-2efc26043fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:59.299298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5891a-3ce7-40b6-adcb-2efc26043fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:59.299572 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5891a-3ce7-40b6-adcb-2efc26043fa6 None admin] PATH_INFO: `/v3/groups/d78e995107bc436bb56514be0b1f1a06` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:59.300771 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf5891a-3ce7-40b6-adcb-2efc26043fa6 None admin] RBAC: Authorizing `identity:get_group(group_id=d78e995107bc436bb56514be0b1f1a06)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:59.302210 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf5891a-3ce7-40b6-adcb-2efc26043fa6 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:59.303794 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 224/454] 10.222.0.20 () {56 vars in 1180 bytes} [Sat Aug 6 12:17:59 2022] GET /identity/v3/groups/d78e995107bc436bb56514be0b1f1a06 => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:17:59.309884 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32565d-c352-477f-a4c0-eb7e3952e070 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:17:59.338996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:17:59.411897 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32565d-c352-477f-a4c0-eb7e3952e070 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:17:59.450582 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc23f8-ddeb-457b-b3cb-86f8864f8815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:17:59.451195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc23f8-ddeb-457b-b3cb-86f8864f8815 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:17:59.451471 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc23f8-ddeb-457b-b3cb-86f8864f8815 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:17:59.451748 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc23f8-ddeb-457b-b3cb-86f8864f8815 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:17:59.452635 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc23f8-ddeb-457b-b3cb-86f8864f8815 None admin] RBAC: Adding query filter params (group.id=d78e995107bc436bb56514be0b1f1a06, role.id=144c8425688b409384e1e8ed8279d88e, scope.project.id=7ad154ab2bd64d7e8e712fcec9e1c94f) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:17:59.452941 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc23f8-ddeb-457b-b3cb-86f8864f8815 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:17:59.454360 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc23f8-ddeb-457b-b3cb-86f8864f8815 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:17:59.460128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 231/455] 10.222.0.20 () {56 vars in 1376 bytes} [Sat Aug 6 12:17:59 2022] GET /identity/v3/role_assignments?group.id=d78e995107bc436bb56514be0b1f1a06&scope.project.id=7ad154ab2bd64d7e8e712fcec9e1c94f&role.id=144c8425688b409384e1e8ed8279d88e => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:01.813729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a432c4-2bb7-4eb2-96bc-0199aa28f1eb None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:01.814604 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a432c4-2bb7-4eb2-96bc-0199aa28f1eb None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:01.815030 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a432c4-2bb7-4eb2-96bc-0199aa28f1eb None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:01.816821 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 225/456] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:01.823335 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deeaf77-fc99-4c96-a54d-f7010168681d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:01.824210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deeaf77-fc99-4c96-a54d-f7010168681d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:01.824705 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deeaf77-fc99-4c96-a54d-f7010168681d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:01.847915 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1deeaf77-fc99-4c96-a54d-f7010168681d None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:01.849450 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1deeaf77-fc99-4c96-a54d-f7010168681d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:01.934958 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 232/457] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:18:01.964872 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deeaf77-fc99-4c96-a54d-f7010168681d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:01.967086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82992ddc-9807-4a47-8aed-4a3ea13e35b2 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:01.967086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82992ddc-9807-4a47-8aed-4a3ea13e35b2 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:01.967086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82992ddc-9807-4a47-8aed-4a3ea13e35b2 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:01.967086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 226/458] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:01.971852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b9dee-95d1-41eb-b08f-603a803e3b96 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:01.972422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b9dee-95d1-41eb-b08f-603a803e3b96 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:01.972901 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b9dee-95d1-41eb-b08f-603a803e3b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:02.005269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:02.005936 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-5c0b9dee-95d1-41eb-b08f-603a803e3b96 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:02.007221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5c0b9dee-95d1-41eb-b08f-603a803e3b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:02.104194 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deeaf77-fc99-4c96-a54d-f7010168681d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:02.119192 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 227/459] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 12:18:02.142910 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b9dee-95d1-41eb-b08f-603a803e3b96 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:02.169539 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2be0c-5c24-4308-a29a-ecbf1e7e9125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:02.170186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2be0c-5c24-4308-a29a-ecbf1e7e9125 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:02.170468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2be0c-5c24-4308-a29a-ecbf1e7e9125 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:02.170738 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2be0c-5c24-4308-a29a-ecbf1e7e9125 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:02.192569 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed2be0c-5c24-4308-a29a-ecbf1e7e9125 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:02.194295 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed2be0c-5c24-4308-a29a-ecbf1e7e9125 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:02.217482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-bed2be0c-5c24-4308-a29a-ecbf1e7e9125 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:02.217482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:02.217482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:02.217482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:02.217482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:02.217482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:02.217482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:02.217482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:02.217482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:02.217482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:02.217482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:02.217482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:02.217482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:02.217482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:02.217482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:02.218370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:02.218370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:02.218370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:02.218370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:02.218370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:02.218370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:02.218370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:02.218370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:02.218370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:02.218370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:02.218370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:02.218370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:02.218370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:02.218370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:02.218370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:02.218370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:02.218370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:02.219197 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:02.219197 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:02.219197 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:02.219197 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:02.221212 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:02.226825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 233/460] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:02.234286 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2be0c-5c24-4308-a29a-ecbf1e7e9125 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:02.315794 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b9dee-95d1-41eb-b08f-603a803e3b96 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:02.376218 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:02.381615 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aea80f-a0cf-4c32-894c-3961b1050ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:02.382272 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aea80f-a0cf-4c32-894c-3961b1050ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:02.382552 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aea80f-a0cf-4c32-894c-3961b1050ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:02.382826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aea80f-a0cf-4c32-894c-3961b1050ff7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:02.409301 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aea80f-a0cf-4c32-894c-3961b1050ff7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:02.410829 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aea80f-a0cf-4c32-894c-3961b1050ff7 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:02.431668 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2be0c-5c24-4308-a29a-ecbf1e7e9125 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:02.440979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-73aea80f-a0cf-4c32-894c-3961b1050ff7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:02.440979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:02.440979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:02.440979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:02.440979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:02.440979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:02.440979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:02.440979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:02.440979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:02.440979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:02.440979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:02.440979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:02.440979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:02.440979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:02.440979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:02.442038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:02.442038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:02.442038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:02.442038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:02.442038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:02.442038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:02.442038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:02.442038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:02.442038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:02.442038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:02.442038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:02.442038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:02.442038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:02.442038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:02.442038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:02.442038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:02.442038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:02.443082 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:02.443082 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:02.443082 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:02.443082 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:02.452291 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 228/461] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:02.453201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aea80f-a0cf-4c32-894c-3961b1050ff7 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:02.496157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38101955-12d6-440e-9a91-4680b34a77c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:02.497047 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38101955-12d6-440e-9a91-4680b34a77c2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:02.497392 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38101955-12d6-440e-9a91-4680b34a77c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:02.497859 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38101955-12d6-440e-9a91-4680b34a77c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:02.498797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38101955-12d6-440e-9a91-4680b34a77c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:02.499339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38101955-12d6-440e-9a91-4680b34a77c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:02.500959 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38101955-12d6-440e-9a91-4680b34a77c2 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:02.514840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 234/462] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:02.516397 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db9f89-7943-4ce8-9085-060bf539fc0e None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:02.516838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db9f89-7943-4ce8-9085-060bf539fc0e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:02.517250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db9f89-7943-4ce8-9085-060bf539fc0e None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:02.518690 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 235/463] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:02.523749 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db9f89-7943-4ce8-9085-060bf539fc0e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:02.575980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:02.578786 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aea80f-a0cf-4c32-894c-3961b1050ff7 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:02.613791 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d705f9-ab8b-44e9-b6ae-ab5b374627af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:02.614627 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d705f9-ab8b-44e9-b6ae-ab5b374627af None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:02.615076 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d705f9-ab8b-44e9-b6ae-ab5b374627af None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:02.615484 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d705f9-ab8b-44e9-b6ae-ab5b374627af None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:02.616608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d705f9-ab8b-44e9-b6ae-ab5b374627af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:02.617064 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d705f9-ab8b-44e9-b6ae-ab5b374627af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:02.618594 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d705f9-ab8b-44e9-b6ae-ab5b374627af None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:02.631580 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 229/464] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:02.637461 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ede22-e5d7-46d1-a4a3-3a0c01e43228 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:02.638015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ede22-e5d7-46d1-a4a3-3a0c01e43228 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:02.638498 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ede22-e5d7-46d1-a4a3-3a0c01e43228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:02.639398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db9f89-7943-4ce8-9085-060bf539fc0e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:02.656773 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-034ede22-e5d7-46d1-a4a3-3a0c01e43228 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:02.658134 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-034ede22-e5d7-46d1-a4a3-3a0c01e43228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:02.676919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53913b-a9ca-4243-ad26-b4b1aaba2bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:02.677535 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53913b-a9ca-4243-ad26-b4b1aaba2bbc None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:02.677815 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53913b-a9ca-4243-ad26-b4b1aaba2bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:02.678090 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53913b-a9ca-4243-ad26-b4b1aaba2bbc None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:02.678761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53913b-a9ca-4243-ad26-b4b1aaba2bbc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:02.680178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53913b-a9ca-4243-ad26-b4b1aaba2bbc None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:02.688690 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:02.703124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 236/465] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Aug 6 12:18:02 2022] POST /identity/v3/users => generated 246 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:18:02.703717 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53913b-a9ca-4243-ad26-b4b1aaba2bbc None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:02.747400 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 230/466] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:18:02.749093 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fdd87-3729-44fd-a4f5-97534204ceaa None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:02.749574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fdd87-3729-44fd-a4f5-97534204ceaa None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:02.750038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fdd87-3729-44fd-a4f5-97534204ceaa None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:02.751633 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 231/467] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:02.757410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5e9f8-a7d0-420a-90bd-c16957f86960 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:02.758326 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5e9f8-a7d0-420a-90bd-c16957f86960 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:02.759428 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5e9f8-a7d0-420a-90bd-c16957f86960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:02.804707 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-23c5e9f8-a7d0-420a-90bd-c16957f86960 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:02.807608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-23c5e9f8-a7d0-420a-90bd-c16957f86960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:02.851719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53913b-a9ca-4243-ad26-b4b1aaba2bbc None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:02.895179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fc027-7ba6-4f81-94fd-1da555b4a6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:02.895724 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fc027-7ba6-4f81-94fd-1da555b4a6fc None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:02.895998 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fc027-7ba6-4f81-94fd-1da555b4a6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:02.896273 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fc027-7ba6-4f81-94fd-1da555b4a6fc None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:02.896982 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412fc027-7ba6-4f81-94fd-1da555b4a6fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:02.898627 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412fc027-7ba6-4f81-94fd-1da555b4a6fc None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:02.904229 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 232/468] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:18:02.904939 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5e9f8-a7d0-420a-90bd-c16957f86960 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:02.926851 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 237/469] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Aug 6 12:18:02 2022] POST /identity/v3/users => generated 248 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:18:02.932890 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:02.940431 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fc027-7ba6-4f81-94fd-1da555b4a6fc None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:03.014658 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:03.044633 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5e9f8-a7d0-420a-90bd-c16957f86960 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:03.083394 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fc027-7ba6-4f81-94fd-1da555b4a6fc None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:03.084970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b41b02-baea-45f5-a23d-6eb60f7e682f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:03.085729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b41b02-baea-45f5-a23d-6eb60f7e682f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:03.086120 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b41b02-baea-45f5-a23d-6eb60f7e682f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:03.086458 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b41b02-baea-45f5-a23d-6eb60f7e682f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:03.098130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b41b02-baea-45f5-a23d-6eb60f7e682f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:03.099721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b41b02-baea-45f5-a23d-6eb60f7e682f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:03.118972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-00b41b02-baea-45f5-a23d-6eb60f7e682f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:03.118972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:03.118972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:03.118972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:03.118972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:03.118972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:03.118972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:03.118972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:03.118972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:03.118972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:03.118972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:03.118972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:03.118972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:03.118972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:03.118972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:03.119884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:03.119884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:03.119884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:03.119884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:03.119884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:03.119884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:03.119884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:03.119884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:03.119884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:03.119884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:03.119884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:03.119884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:03.119884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:03.119884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:03.119884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:03.119884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:03.119884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:03.120769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:03.120769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:03.120769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:03.120769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:03.120769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 233/470] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:03.126608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b41b02-baea-45f5-a23d-6eb60f7e682f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:03.127741 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198cd81-ef6a-497f-84f3-e151053b22e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:03.128631 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198cd81-ef6a-497f-84f3-e151053b22e9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:03.129087 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198cd81-ef6a-497f-84f3-e151053b22e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:03.129558 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198cd81-ef6a-497f-84f3-e151053b22e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:03.143483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0198cd81-ef6a-497f-84f3-e151053b22e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:03.145530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0198cd81-ef6a-497f-84f3-e151053b22e9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:03.162053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-0198cd81-ef6a-497f-84f3-e151053b22e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:03.162053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:03.162053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:03.162053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:03.162053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:03.162053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:03.162053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:03.162053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:03.162053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:03.162053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:03.162053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:03.162053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:03.162053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:03.162053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:03.162053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:03.162944 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:03.162944 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:03.162944 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:03.162944 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:03.162944 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:03.162944 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:03.162944 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:03.162944 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:03.162944 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:03.162944 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:03.162944 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:03.162944 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:03.162944 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:03.162944 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:03.162944 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:03.162944 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:03.162944 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:03.163772 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:03.163772 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:03.163772 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:03.163772 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:03.166613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 238/471] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:03.175517 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198cd81-ef6a-497f-84f3-e151053b22e9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:03.218512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:03.263377 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b41b02-baea-45f5-a23d-6eb60f7e682f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:03.301549 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1342c9ce-c8e9-46c0-be02-c342c4683e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:03.302231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342c9ce-c8e9-46c0-be02-c342c4683e30 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:03.302507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342c9ce-c8e9-46c0-be02-c342c4683e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:03.302782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342c9ce-c8e9-46c0-be02-c342c4683e30 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:03.303750 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1342c9ce-c8e9-46c0-be02-c342c4683e30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:03.303937 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1342c9ce-c8e9-46c0-be02-c342c4683e30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:03.306557 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1342c9ce-c8e9-46c0-be02-c342c4683e30 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:03.313050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:03.319965 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198cd81-ef6a-497f-84f3-e151053b22e9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:03.322652 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 234/472] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:03.328187 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1342c9ce-c8e9-46c0-be02-c342c4683e30 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:03.365848 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff28838-9e73-4eab-80f6-6911fc69e66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:03.367308 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff28838-9e73-4eab-80f6-6911fc69e66d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:03.368180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff28838-9e73-4eab-80f6-6911fc69e66d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:03.369457 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff28838-9e73-4eab-80f6-6911fc69e66d None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:03.370357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff28838-9e73-4eab-80f6-6911fc69e66d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:03.370835 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff28838-9e73-4eab-80f6-6911fc69e66d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:03.373587 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff28838-9e73-4eab-80f6-6911fc69e66d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:03.389600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 239/473] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:03.391150 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65457e9-9493-426e-b45b-29ca078d4876 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:03.391566 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65457e9-9493-426e-b45b-29ca078d4876 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:03.391999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65457e9-9493-426e-b45b-29ca078d4876 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:03.393655 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 240/474] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:03.394643 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65457e9-9493-426e-b45b-29ca078d4876 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:03.446589 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1342c9ce-c8e9-46c0-be02-c342c4683e30 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:03.477631 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:03.484414 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9927d25-d3d6-4ca9-a6c1-b5af0a2f6582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:03.485031 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9927d25-d3d6-4ca9-a6c1-b5af0a2f6582 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:03.485307 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9927d25-d3d6-4ca9-a6c1-b5af0a2f6582 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:03.486001 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9927d25-d3d6-4ca9-a6c1-b5af0a2f6582 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:03.486251 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9927d25-d3d6-4ca9-a6c1-b5af0a2f6582 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:03.487699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9927d25-d3d6-4ca9-a6c1-b5af0a2f6582 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:03.517751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 235/475] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Aug 6 12:18:03 2022] POST /identity/v3/users => generated 248 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 12:18:03.519104 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84698f0-cee5-47c7-b913-29b58523a510 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:03.519444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84698f0-cee5-47c7-b913-29b58523a510 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:03.519729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84698f0-cee5-47c7-b913-29b58523a510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:03.536773 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65457e9-9493-426e-b45b-29ca078d4876 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:03.540909 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-f84698f0-cee5-47c7-b913-29b58523a510 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:03.541961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f84698f0-cee5-47c7-b913-29b58523a510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:03.600518 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9a2fd-9349-4ad7-96d0-2176b80aaf44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:03.601496 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9a2fd-9349-4ad7-96d0-2176b80aaf44 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:03.601997 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9a2fd-9349-4ad7-96d0-2176b80aaf44 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:03.602465 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d9a2fd-9349-4ad7-96d0-2176b80aaf44 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:03.603419 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d9a2fd-9349-4ad7-96d0-2176b80aaf44 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:03.605076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d9a2fd-9349-4ad7-96d0-2176b80aaf44 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:03.641299 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 241/476] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Aug 6 12:18:03 2022] POST /identity/v3/users => generated 249 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 12:18:03.655432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 236/477] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:18:03.693111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9a2fd-9349-4ad7-96d0-2176b80aaf44 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:03.714451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:03.817284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d9a2fd-9349-4ad7-96d0-2176b80aaf44 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:03.858128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983b0bf-69dd-4762-8633-59ab7540ad3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:03.858736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983b0bf-69dd-4762-8633-59ab7540ad3c None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:03.859014 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983b0bf-69dd-4762-8633-59ab7540ad3c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:03.859285 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983b0bf-69dd-4762-8633-59ab7540ad3c None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:03.860004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b983b0bf-69dd-4762-8633-59ab7540ad3c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:03.861418 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b983b0bf-69dd-4762-8633-59ab7540ad3c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:03.871827 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.sql.core [None req-b983b0bf-69dd-4762-8633-59ab7540ad3c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 06 12:18:03.871827 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 06 12:18:03.871827 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [parameters: {'id': '3edd37f955cc4a3b92135278ec57d4de', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 06 12:18:03.871827 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80144) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 06 12:18:03.889435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-b983b0bf-69dd-4762-8633-59ab7540ad3c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 06 12:18:03.889435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:03.889435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 12:18:03.889435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 12:18:03.889435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 12:18:03.889435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 12:18:03.889435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 12:18:03.889435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 12:18:03.889435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 12:18:03.889435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application conn.query(q) Aug 06 12:18:03.889435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 12:18:03.889435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 12:18:03.889435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 12:18:03.889435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application result.read() Aug 06 12:18:03.889435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 12:18:03.890339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 12:18:03.890339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 12:18:03.890339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 12:18:03.890339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 12:18:03.890339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 12:18:03.890339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 12:18:03.890339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 12:18:03.890339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 06 12:18:03.890339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:03.890339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 06 12:18:03.890339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:03.890339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:03.890339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 06 12:18:03.890339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 06 12:18:03.890339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 06 12:18:03.890339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return ref.to_dict() Aug 06 12:18:03.890339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 12:18:03.890339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application next(self.gen) Aug 06 12:18:03.891259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 06 12:18:03.891259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application yield resource Aug 06 12:18:03.891259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 12:18:03.891259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application next(self.gen) Aug 06 12:18:03.891259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 06 12:18:03.891259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.session.rollback() Aug 06 12:18:03.891259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 06 12:18:03.891259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.force_reraise() Aug 06 12:18:03.891259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 06 12:18:03.891259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise self.value Aug 06 12:18:03.891259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 06 12:18:03.891259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 06 12:18:03.891259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 06 12:18:03.891259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application session.commit() Aug 06 12:18:03.891259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 06 12:18:03.891259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 06 12:18:03.891259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 06 12:18:03.892108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._prepare_impl() Aug 06 12:18:03.892108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 06 12:18:03.892108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.session.flush() Aug 06 12:18:03.892108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 06 12:18:03.892108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._flush(objects) Aug 06 12:18:03.892108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 06 12:18:03.892108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 06 12:18:03.892108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 06 12:18:03.892108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application compat.raise_( Aug 06 12:18:03.892108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 12:18:03.892108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception Aug 06 12:18:03.892108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 06 12:18:03.892108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application flush_context.execute() Aug 06 12:18:03.892108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 06 12:18:03.892108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rec.execute(self) Aug 06 12:18:03.892108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 06 12:18:03.892108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 06 12:18:03.893033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 06 12:18:03.893033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 06 12:18:03.893033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 06 12:18:03.893033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 06 12:18:03.893033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 06 12:18:03.893033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 06 12:18:03.893033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 06 12:18:03.893033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 06 12:18:03.893033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 06 12:18:03.893033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 06 12:18:03.893033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 06 12:18:03.893033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 06 12:18:03.893033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 06 12:18:03.893033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 06 12:18:03.893033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 12:18:03.893033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception Aug 06 12:18:03.893849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 12:18:03.893849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 12:18:03.893849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 12:18:03.893849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 12:18:03.893849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 12:18:03.893849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 12:18:03.893849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 12:18:03.893849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application conn.query(q) Aug 06 12:18:03.893849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 12:18:03.893849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 12:18:03.893849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 12:18:03.893849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application result.read() Aug 06 12:18:03.893849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 12:18:03.893849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 12:18:03.893849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 12:18:03.893849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 12:18:03.894639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 12:18:03.894639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 12:18:03.894639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 12:18:03.894639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 12:18:03.894639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 06 12:18:03.894639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 06 12:18:03.894639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [parameters: {'id': '3edd37f955cc4a3b92135278ec57d4de', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 06 12:18:03.894639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 06 12:18:03.894639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:03.894639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:18:03.894639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:03.894639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:03.894639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:03.894639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:03.894639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:03.894639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:03.895440 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:03.895440 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:03.895440 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:03.895440 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:03.895440 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:03.895440 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:03.895440 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 06 12:18:03.895440 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 06 12:18:03.895440 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:03.895440 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:03.895440 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 06 12:18:03.895440 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 06 12:18:03.895440 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 06 12:18:03.895440 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 06 12:18:03.895440 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 06 12:18:03.895440 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:03.896282 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 242/478] 10.222.0.20 () {60 vars in 1131 bytes} [Sat Aug 6 12:18:03 2022] POST /identity/v3/roles => generated 144 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:03.900866 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84698f0-cee5-47c7-b913-29b58523a510 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:04.019835 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84698f0-cee5-47c7-b913-29b58523a510 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:04.045092 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:04.066976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79f4bd-0382-4f4e-ade1-d68baca7b632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:04.067621 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79f4bd-0382-4f4e-ade1-d68baca7b632 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:04.067896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79f4bd-0382-4f4e-ade1-d68baca7b632 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:04.068231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79f4bd-0382-4f4e-ade1-d68baca7b632 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:04.085404 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79f4bd-0382-4f4e-ade1-d68baca7b632 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:04.090608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-5d79f4bd-0382-4f4e-ade1-d68baca7b632 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:18:04.090608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:04.090608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:04.090608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:04.090608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:04.090608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:04.090608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:04.090608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:04.090608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:04.090608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:04.090608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:04.090608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:04.090608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:04.090608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:04.090608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:04.090608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:04.092735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:04.092735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:04.092735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:04.092735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:04.092735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:04.092735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:04.092735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:04.092735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:04.092735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:04.092735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:04.092735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:04.092735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:04.092735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:04.092735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:04.092735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:04.092735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:04.094671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:04.094671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:04.094671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:04.094671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:04.094671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:04.094671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:04.094671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:04.094671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:04.094671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:04.094671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:04.094671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 06 12:18:04.094671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:04.100321 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 237/479] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Aug 6 12:18:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:04.100507 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983b0bf-69dd-4762-8633-59ab7540ad3c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:04.244156 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:04.275658 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983b0bf-69dd-4762-8633-59ab7540ad3c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:04.314452 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd564bd-d07b-4a80-876a-406a6f5bba6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:04.315069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd564bd-d07b-4a80-876a-406a6f5bba6a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:04.315344 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd564bd-d07b-4a80-876a-406a6f5bba6a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:04.315615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd564bd-d07b-4a80-876a-406a6f5bba6a None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:04.316351 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd564bd-d07b-4a80-876a-406a6f5bba6a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:04.316655 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd564bd-d07b-4a80-876a-406a6f5bba6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:04.318014 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd564bd-d07b-4a80-876a-406a6f5bba6a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:04.330704 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 243/480] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:05.518719 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b20d2-a557-4437-8ce9-0fc89be960e5 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:05.519236 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b20d2-a557-4437-8ce9-0fc89be960e5 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:05.519236 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b20d2-a557-4437-8ce9-0fc89be960e5 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:05.520915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 238/481] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:05.526141 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb262d87-380e-4e83-995f-19154d201c94 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:05.526633 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb262d87-380e-4e83-995f-19154d201c94 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:05.526786 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb262d87-380e-4e83-995f-19154d201c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:05.545486 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-eb262d87-380e-4e83-995f-19154d201c94 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:05.546658 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-eb262d87-380e-4e83-995f-19154d201c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:05.560765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f0771-9c70-4e68-ba02-50a72ca108ae None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:05.561073 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f0771-9c70-4e68-ba02-50a72ca108ae None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:05.561311 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f0771-9c70-4e68-ba02-50a72ca108ae None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:05.563348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 239/482] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:18:05.569690 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a01a2-cab5-4051-a76d-e87a8ca03cf3 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:05.569971 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a01a2-cab5-4051-a76d-e87a8ca03cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:05.570244 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a01a2-cab5-4051-a76d-e87a8ca03cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:05.599974 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-a85a01a2-cab5-4051-a76d-e87a8ca03cf3 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:05.601210 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a85a01a2-cab5-4051-a76d-e87a8ca03cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:05.693520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 244/483] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:18:05.724512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 240/484] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 12:18:05.732322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb262d87-380e-4e83-995f-19154d201c94 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:05.805760 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a01a2-cab5-4051-a76d-e87a8ca03cf3 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:05.822407 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:05.872358 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb262d87-380e-4e83-995f-19154d201c94 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:05.947387 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e172402-a100-4e01-a2d8-f3c7a4cc3908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:05.948044 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e172402-a100-4e01-a2d8-f3c7a4cc3908 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:05.948335 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e172402-a100-4e01-a2d8-f3c7a4cc3908 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:05.948637 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e172402-a100-4e01-a2d8-f3c7a4cc3908 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:05.959717 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e172402-a100-4e01-a2d8-f3c7a4cc3908 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:05.967333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-1e172402-a100-4e01-a2d8-f3c7a4cc3908 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:05.967333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:05.967333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:05.967333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:05.967333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:05.967333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:05.967333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:05.967333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:05.967333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:05.967333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:05.967333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:05.967333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:05.967333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:05.967333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:05.967333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:05.967333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:05.968448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:05.968448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:05.968448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:05.968448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:05.968448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:05.968448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:05.968448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:05.968448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:05.968448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:05.968448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:05.968448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:05.968448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:05.968448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:05.968448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:05.968448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:05.968448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:05.969310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:05.969310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:05.969310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:05.969310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:05.969310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:05.969310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:05.969310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:05.969310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:05.969310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:05.969310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:05.969310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:05.969310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:05.969310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 245/485] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Aug 6 12:18:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:05.970036 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d208c-1c71-4e7c-a17a-7a8ca51d4381 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:05.970267 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d208c-1c71-4e7c-a17a-7a8ca51d4381 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:05.970543 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d208c-1c71-4e7c-a17a-7a8ca51d4381 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:05.972422 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 246/486] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:05.977190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d208c-1c71-4e7c-a17a-7a8ca51d4381 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:05.995839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a01a2-cab5-4051-a76d-e87a8ca03cf3 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:06.007246 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:06.034058 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda70b24-95af-4013-95a4-7e1e8f2f8735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:06.034915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda70b24-95af-4013-95a4-7e1e8f2f8735 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:06.035332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda70b24-95af-4013-95a4-7e1e8f2f8735 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:06.035739 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda70b24-95af-4013-95a4-7e1e8f2f8735 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:06.050371 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda70b24-95af-4013-95a4-7e1e8f2f8735 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:06.055980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-eda70b24-95af-4013-95a4-7e1e8f2f8735 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:06.055980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:06.055980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:06.055980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:06.055980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:06.055980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:06.055980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:06.055980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:06.055980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:06.055980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:06.055980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:06.055980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:06.055980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:06.055980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:06.055980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:06.055980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:06.057464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:06.057464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:06.057464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:06.057464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:06.057464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:06.057464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:06.057464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:06.057464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:06.057464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:06.057464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:06.057464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:06.057464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:06.057464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:06.057464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:06.057464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:06.057464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:06.058253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:06.058253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:06.058253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:06.058253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:06.058253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:06.058253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:06.058253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:06.058253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:06.058253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:06.058253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:06.058253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:06.058253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:06.061118 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 241/487] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Aug 6 12:18:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:06.062495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a45f13-b2b0-43a3-a861-f4d75bea9da1 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:06.062991 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a45f13-b2b0-43a3-a861-f4d75bea9da1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:06.063457 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a45f13-b2b0-43a3-a861-f4d75bea9da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:06.083163 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-01a45f13-b2b0-43a3-a861-f4d75bea9da1 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:06.084857 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-01a45f13-b2b0-43a3-a861-f4d75bea9da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:06.117959 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d208c-1c71-4e7c-a17a-7a8ca51d4381 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:06.154981 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:06.160551 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 242/488] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:18:06.161389 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a45f13-b2b0-43a3-a861-f4d75bea9da1 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:06.162310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf18dd-735b-4c8b-b9a0-657cc328b9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:06.162928 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf18dd-735b-4c8b-b9a0-657cc328b9cd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:06.163208 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf18dd-735b-4c8b-b9a0-657cc328b9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:06.163757 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf18dd-735b-4c8b-b9a0-657cc328b9cd None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:06.165429 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf18dd-735b-4c8b-b9a0-657cc328b9cd None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:06.165739 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf18dd-735b-4c8b-b9a0-657cc328b9cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:06.167103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf18dd-735b-4c8b-b9a0-657cc328b9cd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:06.177600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 247/489] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Aug 6 12:18:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:06.179081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b0d1d-8838-4cb3-b5e6-36a01a19f65b None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:06.179368 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b0d1d-8838-4cb3-b5e6-36a01a19f65b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:06.179645 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b0d1d-8838-4cb3-b5e6-36a01a19f65b None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:06.185142 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 248/490] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:06.186418 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b0d1d-8838-4cb3-b5e6-36a01a19f65b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:06.278813 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:06.282777 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a45f13-b2b0-43a3-a861-f4d75bea9da1 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:06.305642 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b0d1d-8838-4cb3-b5e6-36a01a19f65b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:06.320257 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93641839-ad17-487a-8170-9600cb16a617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:06.320832 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93641839-ad17-487a-8170-9600cb16a617 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:06.321125 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93641839-ad17-487a-8170-9600cb16a617 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:06.321377 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93641839-ad17-487a-8170-9600cb16a617 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:06.322861 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93641839-ad17-487a-8170-9600cb16a617 None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:06.323177 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93641839-ad17-487a-8170-9600cb16a617 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:06.324612 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93641839-ad17-487a-8170-9600cb16a617 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:06.332625 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 243/491] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Aug 6 12:18:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:06.333895 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d114245-17e7-40b4-98ee-306eb2981460 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:06.334252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d114245-17e7-40b4-98ee-306eb2981460 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:06.334611 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d114245-17e7-40b4-98ee-306eb2981460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:06.348543 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf61a1d3-0e03-43dd-adbc-054f9f7fc6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:06.349176 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61a1d3-0e03-43dd-adbc-054f9f7fc6bc None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:06.349451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61a1d3-0e03-43dd-adbc-054f9f7fc6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:06.349972 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf61a1d3-0e03-43dd-adbc-054f9f7fc6bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:06.353827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-7d114245-17e7-40b4-98ee-306eb2981460 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:06.354994 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7d114245-17e7-40b4-98ee-306eb2981460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:06.361922 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf61a1d3-0e03-43dd-adbc-054f9f7fc6bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:06.363487 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf61a1d3-0e03-43dd-adbc-054f9f7fc6bc None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:06.379005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-bf61a1d3-0e03-43dd-adbc-054f9f7fc6bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:06.379005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:06.379005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:06.379005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:06.379005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:06.379005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:06.379005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:06.379005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:06.379005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:06.379005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:06.379005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:06.379005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:06.379005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:06.379005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:06.379005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:06.379838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:06.379838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:06.379838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:06.379838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:06.379838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:06.379838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:06.379838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:06.379838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:06.379838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:06.379838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:06.379838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:06.379838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:06.379838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:06.379838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:06.379838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:06.379838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:06.379838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:06.380715 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:06.380715 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:06.380715 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:06.380715 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:06.380715 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 249/492] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:06.380984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf61a1d3-0e03-43dd-adbc-054f9f7fc6bc None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:06.427861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:06.435197 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 244/493] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:18:06.440921 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d114245-17e7-40b4-98ee-306eb2981460 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:06.491294 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf61a1d3-0e03-43dd-adbc-054f9f7fc6bc None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:06.530945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2b199-fd76-4ca7-b5a9-b5300fc9d123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:06.532205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:06.532774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2b199-fd76-4ca7-b5a9-b5300fc9d123 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:06.533214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2b199-fd76-4ca7-b5a9-b5300fc9d123 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:06.533939 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2b199-fd76-4ca7-b5a9-b5300fc9d123 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:06.543232 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d114245-17e7-40b4-98ee-306eb2981460 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:06.546628 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2b199-fd76-4ca7-b5a9-b5300fc9d123 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:06.552334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-43a2b199-fd76-4ca7-b5a9-b5300fc9d123 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:06.552334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:06.552334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:06.552334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:06.552334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:06.552334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:06.552334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:06.552334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:06.552334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:06.552334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:06.552334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:06.552334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:06.552334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:06.552334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:06.552334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:06.552334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:06.553221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:06.553221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:06.553221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:06.553221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:06.553221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:06.553221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:06.553221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:06.553221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:06.553221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:06.553221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:06.553221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:06.553221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:06.553221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:06.553221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:06.553221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:06.553221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:06.554009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:06.554009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:06.554009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:06.554009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:06.554009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:06.554009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:06.554009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:06.554009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:06.554009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:06.554009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:06.554009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:06.554009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:06.556784 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 250/494] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Aug 6 12:18:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:06.557793 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2b199-fd76-4ca7-b5a9-b5300fc9d123 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:06.584648 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640babd-8630-4ba1-bde0-86d3c7324d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:06.585205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640babd-8630-4ba1-bde0-86d3c7324d3d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:06.585483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640babd-8630-4ba1-bde0-86d3c7324d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:06.585757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640babd-8630-4ba1-bde0-86d3c7324d3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:06.597824 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b640babd-8630-4ba1-bde0-86d3c7324d3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:06.599283 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b640babd-8630-4ba1-bde0-86d3c7324d3d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:06.615520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-b640babd-8630-4ba1-bde0-86d3c7324d3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:06.615520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:06.615520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:06.615520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:06.615520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:06.615520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:06.615520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:06.615520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:06.615520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:06.615520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:06.615520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:06.615520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:06.615520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:06.615520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:06.615520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:06.617475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:06.617475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:06.617475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:06.617475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:06.617475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:06.617475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:06.617475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:06.617475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:06.617475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:06.617475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:06.617475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:06.617475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:06.617475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:06.617475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:06.617475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:06.617475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:06.617475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:06.618908 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:06.618908 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:06.618908 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:06.618908 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:06.622015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 245/495] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:06.622771 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640babd-8630-4ba1-bde0-86d3c7324d3d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:06.686369 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2b199-fd76-4ca7-b5a9-b5300fc9d123 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:06.689550 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:06.723342 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e1ce5-12e6-44fb-b227-780026465337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:06.723957 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e1ce5-12e6-44fb-b227-780026465337 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:06.724231 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e1ce5-12e6-44fb-b227-780026465337 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:06.724534 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e1ce5-12e6-44fb-b227-780026465337 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:06.725215 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e1ce5-12e6-44fb-b227-780026465337 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:06.725513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e1ce5-12e6-44fb-b227-780026465337 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:06.726908 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e1ce5-12e6-44fb-b227-780026465337 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:06.730296 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640babd-8630-4ba1-bde0-86d3c7324d3d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:06.739755 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 251/496] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:06.740677 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e1ce5-12e6-44fb-b227-780026465337 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:06.767777 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe9bba-f5b6-4051-944b-43bf79b60000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:06.768809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe9bba-f5b6-4051-944b-43bf79b60000 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:06.769325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe9bba-f5b6-4051-944b-43bf79b60000 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:06.769669 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe9bba-f5b6-4051-944b-43bf79b60000 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:06.782820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe9bba-f5b6-4051-944b-43bf79b60000 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:06.789356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-cdbe9bba-f5b6-4051-944b-43bf79b60000 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:06.789356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:06.789356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:06.789356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:06.789356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:06.789356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:06.789356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:06.789356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:06.789356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:06.789356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:06.789356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:06.789356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:06.789356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:06.789356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:06.789356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:06.789356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:06.790221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:06.790221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:06.790221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:06.790221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:06.790221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:06.790221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:06.790221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:06.790221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:06.790221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:06.790221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:06.790221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:06.790221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:06.790221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:06.790221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:06.790221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:06.790221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:06.792235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:06.792235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:06.792235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:06.792235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:06.792235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:06.792235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:06.792235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:06.792235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:06.792235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:06.792235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:06.792235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:06.792235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:06.792235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 246/497] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Aug 6 12:18:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:06.792235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe9bba-f5b6-4051-944b-43bf79b60000 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:06.794942 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:06.854946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e1ce5-12e6-44fb-b227-780026465337 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:06.890270 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db486ed3-9fd7-4e8d-a8d3-79d9d55c0ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:06.890880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db486ed3-9fd7-4e8d-a8d3-79d9d55c0ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:06.891157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db486ed3-9fd7-4e8d-a8d3-79d9d55c0ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:06.891426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db486ed3-9fd7-4e8d-a8d3-79d9d55c0ba8 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:06.892140 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db486ed3-9fd7-4e8d-a8d3-79d9d55c0ba8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:06.892537 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db486ed3-9fd7-4e8d-a8d3-79d9d55c0ba8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:06.893907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db486ed3-9fd7-4e8d-a8d3-79d9d55c0ba8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:06.899590 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe9bba-f5b6-4051-944b-43bf79b60000 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:06.907035 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 252/498] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Aug 6 12:18:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:06.907946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db486ed3-9fd7-4e8d-a8d3-79d9d55c0ba8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:06.938668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62755d5e-322d-4234-9375-fa3fec7c6918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:06.939287 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62755d5e-322d-4234-9375-fa3fec7c6918 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:06.939568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62755d5e-322d-4234-9375-fa3fec7c6918 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:06.939847 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62755d5e-322d-4234-9375-fa3fec7c6918 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:06.940579 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62755d5e-322d-4234-9375-fa3fec7c6918 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:06.940885 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62755d5e-322d-4234-9375-fa3fec7c6918 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:06.941604 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:06.942275 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62755d5e-322d-4234-9375-fa3fec7c6918 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:06.955011 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 247/499] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:06.957178 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62755d5e-322d-4234-9375-fa3fec7c6918 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:07.020139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db486ed3-9fd7-4e8d-a8d3-79d9d55c0ba8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:07.021881 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:07.053533 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaeab62-41de-4d88-8f9b-7d00f8172251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:07.054169 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaeab62-41de-4d88-8f9b-7d00f8172251 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:07.054446 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaeab62-41de-4d88-8f9b-7d00f8172251 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:07.054719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaeab62-41de-4d88-8f9b-7d00f8172251 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:07.059944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62755d5e-322d-4234-9375-fa3fec7c6918 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:07.062715 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaeab62-41de-4d88-8f9b-7d00f8172251 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:07.064135 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaeab62-41de-4d88-8f9b-7d00f8172251 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:07.076490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-ccaeab62-41de-4d88-8f9b-7d00f8172251 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 06 12:18:07.076490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:07.076490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:07.076490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:07.076490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:07.076490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:07.076490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:07.076490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:07.076490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:07.076490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:07.076490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:07.076490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:07.076490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:07.076490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:07.076490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:07.076490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:07.077379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:07.077379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:07.077379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:07.077379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:07.077379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:07.077379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:07.077379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:07.077379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:07.077379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:07.077379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:07.077379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:07.077379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:07.077379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:07.077379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:07.077379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:07.077379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:07.077379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:07.077379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:07.078682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:07.078682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:07.078682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:07.078682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:07.078682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:07.078682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:07.078682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:07.078682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:07.078682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 06 12:18:07.078682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:07.078682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 253/500] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:18:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:07.078682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaeab62-41de-4d88-8f9b-7d00f8172251 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:07.097019 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d2e44-2145-4509-b07b-d1ddbacaaaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:07.097622 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d2e44-2145-4509-b07b-d1ddbacaaaed None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:07.097901 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d2e44-2145-4509-b07b-d1ddbacaaaed None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:07.098209 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d2e44-2145-4509-b07b-d1ddbacaaaed None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:07.098913 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d2e44-2145-4509-b07b-d1ddbacaaaed None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:07.099215 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d2e44-2145-4509-b07b-d1ddbacaaaed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:07.100621 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d2e44-2145-4509-b07b-d1ddbacaaaed None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:07.108847 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 248/501] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Aug 6 12:18:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:07.109558 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d2e44-2145-4509-b07b-d1ddbacaaaed None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:07.162037 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:07.177677 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaeab62-41de-4d88-8f9b-7d00f8172251 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:07.209918 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e436695-71c6-4e0d-b0d7-3404afda0095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:07.210550 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e436695-71c6-4e0d-b0d7-3404afda0095 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:07.210650 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d2e44-2145-4509-b07b-d1ddbacaaaed None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:07.210851 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e436695-71c6-4e0d-b0d7-3404afda0095 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:07.211110 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e436695-71c6-4e0d-b0d7-3404afda0095 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:07.221961 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e436695-71c6-4e0d-b0d7-3404afda0095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:07.223401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e436695-71c6-4e0d-b0d7-3404afda0095 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:07.232920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:07.237802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-9e436695-71c6-4e0d-b0d7-3404afda0095 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:07.237802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:07.237802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:07.237802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:07.237802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:07.237802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:07.237802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:07.237802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:07.237802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:07.237802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:07.237802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:07.237802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:07.237802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:07.237802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:07.237802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:07.238660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:07.238660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:07.238660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:07.238660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:07.238660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:07.238660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:07.238660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:07.238660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:07.238660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:07.238660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:07.238660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:07.238660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:07.238660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:07.238660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:07.238660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:07.238660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:07.238660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:07.239607 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:07.239607 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:07.239607 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:07.239607 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:07.239607 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 254/502] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:07.240184 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283461eb-c32e-4a35-8076-2d9c24f24d51 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:07.240494 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283461eb-c32e-4a35-8076-2d9c24f24d51 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:07.240766 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283461eb-c32e-4a35-8076-2d9c24f24d51 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:07.242277 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 255/503] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:07.242969 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283461eb-c32e-4a35-8076-2d9c24f24d51 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:07.251126 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670472c-931c-476b-85f1-0312334a8081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:07.251981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670472c-931c-476b-85f1-0312334a8081 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:07.252250 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670472c-931c-476b-85f1-0312334a8081 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:07.252549 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670472c-931c-476b-85f1-0312334a8081 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:07.262010 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2670472c-931c-476b-85f1-0312334a8081 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:07.263659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2670472c-931c-476b-85f1-0312334a8081 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:07.276647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-2670472c-931c-476b-85f1-0312334a8081 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 12:18:07.276647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:07.276647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:07.276647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:07.276647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:07.276647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:07.276647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:07.276647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:07.276647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:07.276647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:07.276647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:07.276647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:07.276647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:07.276647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:07.276647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:07.276647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:07.277716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:07.277716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:07.277716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:07.277716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:07.277716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:07.277716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:07.277716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:07.277716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:07.277716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:07.277716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:07.277716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:07.277716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:07.277716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:07.277716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:07.277716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:07.277716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:07.277716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:07.277716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:07.278614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:07.278614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:07.278614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:07.278614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:07.278614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:07.278614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:07.278614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:07.278614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:07.278614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 12:18:07.278614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:07.278614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 249/504] 10.222.0.20 () {56 vars in 1093 bytes} [Sat Aug 6 12:18:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:07.279174 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670472c-931c-476b-85f1-0312334a8081 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:07.346266 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283461eb-c32e-4a35-8076-2d9c24f24d51 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:07.376927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670472c-931c-476b-85f1-0312334a8081 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:07.378914 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa97184-610b-475f-a47a-eb80da10e9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:07.379519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa97184-610b-475f-a47a-eb80da10e9be None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:07.379794 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa97184-610b-475f-a47a-eb80da10e9be None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:07.380062 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa97184-610b-475f-a47a-eb80da10e9be None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:07.380878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa97184-610b-475f-a47a-eb80da10e9be None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:07.381183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa97184-610b-475f-a47a-eb80da10e9be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:07.382604 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa97184-610b-475f-a47a-eb80da10e9be None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:07.400042 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:07.409852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a6e2c-46d1-4dd4-afee-538ba4c31a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:07.410479 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a6e2c-46d1-4dd4-afee-538ba4c31a72 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:07.410760 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a6e2c-46d1-4dd4-afee-538ba4c31a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:07.411034 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a6e2c-46d1-4dd4-afee-538ba4c31a72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:07.419504 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 256/505] 10.222.0.20 () {56 vars in 1137 bytes} [Sat Aug 6 12:18:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:07.420320 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa97184-610b-475f-a47a-eb80da10e9be None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:07.426019 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a6e2c-46d1-4dd4-afee-538ba4c31a72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:07.427487 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a6e2c-46d1-4dd4-afee-538ba4c31a72 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:07.442086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-b70a6e2c-46d1-4dd4-afee-538ba4c31a72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:07.442086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:07.442086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:07.442086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:07.442086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:07.442086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:07.442086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:07.442086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:07.442086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:07.442086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:07.442086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:07.442086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:07.442086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:07.442086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:07.442086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:07.442927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:07.442927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:07.442927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:07.442927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:07.442927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:07.442927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:07.442927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:07.442927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:07.442927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:07.442927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:07.442927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:07.442927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:07.442927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:07.442927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:07.442927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:07.442927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:07.442927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:07.443800 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:07.443800 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:07.443800 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:07.443800 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:07.443800 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 250/506] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:07.444682 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3a489-e16e-4df1-b3a5-77b42a9c20c5 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:07.444966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3a489-e16e-4df1-b3a5-77b42a9c20c5 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:07.445240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3a489-e16e-4df1-b3a5-77b42a9c20c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:07.462762 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-93f3a489-e16e-4df1-b3a5-77b42a9c20c5 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:07.463971 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-93f3a489-e16e-4df1-b3a5-77b42a9c20c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:07.491381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:07.522718 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa97184-610b-475f-a47a-eb80da10e9be None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:07.530300 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 251/507] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:18:07.531101 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3a489-e16e-4df1-b3a5-77b42a9c20c5 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:07.557601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535c587-6c0a-4396-8f63-880d6a09acdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:07.558281 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535c587-6c0a-4396-8f63-880d6a09acdb None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:07.558472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535c587-6c0a-4396-8f63-880d6a09acdb None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:07.558742 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535c587-6c0a-4396-8f63-880d6a09acdb None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:07.559426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535c587-6c0a-4396-8f63-880d6a09acdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:07.559725 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535c587-6c0a-4396-8f63-880d6a09acdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:07.561125 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535c587-6c0a-4396-8f63-880d6a09acdb None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:07.575676 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 257/508] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:07.576560 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535c587-6c0a-4396-8f63-880d6a09acdb None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:07.639400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:07.650418 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3a489-e16e-4df1-b3a5-77b42a9c20c5 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:07.682647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535c587-6c0a-4396-8f63-880d6a09acdb None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:07.683368 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2982f96-bc92-4f5c-b29c-15646d411c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:07.683979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2982f96-bc92-4f5c-b29c-15646d411c60 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:07.684260 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2982f96-bc92-4f5c-b29c-15646d411c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:07.684563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2982f96-bc92-4f5c-b29c-15646d411c60 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:07.685348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2982f96-bc92-4f5c-b29c-15646d411c60 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:07.685655 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2982f96-bc92-4f5c-b29c-15646d411c60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:07.687096 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2982f96-bc92-4f5c-b29c-15646d411c60 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:07.715764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15a602-d33a-4a10-8a69-1cff2443dbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:07.716412 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15a602-d33a-4a10-8a69-1cff2443dbde None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:07.716693 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15a602-d33a-4a10-8a69-1cff2443dbde None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:07.716964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15a602-d33a-4a10-8a69-1cff2443dbde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:07.724043 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 252/509] 10.222.0.20 () {56 vars in 1133 bytes} [Sat Aug 6 12:18:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:07.724896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2982f96-bc92-4f5c-b29c-15646d411c60 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:07.748779 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:07.787703 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15a602-d33a-4a10-8a69-1cff2443dbde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:07.789166 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15a602-d33a-4a10-8a69-1cff2443dbde None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:07.823250 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2982f96-bc92-4f5c-b29c-15646d411c60 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:07.842873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:07.856090 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50aa5f-120d-4bc8-b391-ea009c58effe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:07.856724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50aa5f-120d-4bc8-b391-ea009c58effe None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:07.856999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50aa5f-120d-4bc8-b391-ea009c58effe None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:07.857266 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50aa5f-120d-4bc8-b391-ea009c58effe None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:07.857943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50aa5f-120d-4bc8-b391-ea009c58effe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:07.858244 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50aa5f-120d-4bc8-b391-ea009c58effe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:07.859604 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50aa5f-120d-4bc8-b391-ea009c58effe None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:07.870713 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 253/510] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:07.871479 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50aa5f-120d-4bc8-b391-ea009c58effe None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:07.891801 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 258/511] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:18:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:18:07.891801 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15a602-d33a-4a10-8a69-1cff2443dbde None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:07.967250 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:07.984465 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50aa5f-120d-4bc8-b391-ea009c58effe None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:07.992825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15a602-d33a-4a10-8a69-1cff2443dbde None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:08.018795 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6340c-4320-4e1c-ac59-d3904bd6e2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:08.019423 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6340c-4320-4e1c-ac59-d3904bd6e2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:08.019703 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6340c-4320-4e1c-ac59-d3904bd6e2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:08.019977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6340c-4320-4e1c-ac59-d3904bd6e2f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:08.026108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ca61b-5046-4e35-a1a8-66921414427b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:08.026728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ca61b-5046-4e35-a1a8-66921414427b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:08.027003 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ca61b-5046-4e35-a1a8-66921414427b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:08.027274 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ca61b-5046-4e35-a1a8-66921414427b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:08.030802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6340c-4320-4e1c-ac59-d3904bd6e2f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:08.032252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6340c-4320-4e1c-ac59-d3904bd6e2f0 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:08.034087 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ca61b-5046-4e35-a1a8-66921414427b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:08.035494 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ca61b-5046-4e35-a1a8-66921414427b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:08.046173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-dbf6340c-4320-4e1c-ac59-d3904bd6e2f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:08.046173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:08.046173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:08.046173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:08.046173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:08.046173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:08.046173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:08.046173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:08.046173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:08.046173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:08.046173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:08.046173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:08.046173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:08.046173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:08.046173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:08.046986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:08.046986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:08.046986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:08.046986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:08.046986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:08.046986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:08.046986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:08.046986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:08.046986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:08.046986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:08.046986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:08.046986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:08.046986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:08.046986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:08.046986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:08.046986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:08.046986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:08.047822 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:08.047822 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:08.047822 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:08.047822 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:08.047822 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 254/512] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:08.048123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-442ca61b-5046-4e35-a1a8-66921414427b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 12:18:08.048123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:08.048123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:08.048123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:08.048123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:08.048123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:08.048123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:08.048123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:08.048123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:08.048123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:08.048123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:08.048123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:08.048123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:08.048123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:08.048123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:08.048123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:08.049666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:08.049666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:08.049666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:08.049666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:08.049666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:08.049666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:08.049666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:08.049666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:08.049666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:08.049666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:08.049666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:08.049666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:08.049666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:08.049666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:08.049666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:08.049666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:08.049666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:08.049666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:08.050968 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:08.050968 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:08.050968 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:08.050968 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:08.050968 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:08.050968 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:08.050968 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:08.050968 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:08.050968 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 12:18:08.050968 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:08.051579 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6340c-4320-4e1c-ac59-d3904bd6e2f0 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:08.051659 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 259/513] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Aug 6 12:18:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:08.051659 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ca61b-5046-4e35-a1a8-66921414427b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:08.070319 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:08.149686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6340c-4320-4e1c-ac59-d3904bd6e2f0 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:08.151277 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ca61b-5046-4e35-a1a8-66921414427b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:08.182285 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7b496-7093-47de-ba0c-9f1bce379082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:08.182887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7b496-7093-47de-ba0c-9f1bce379082 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:08.183161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7b496-7093-47de-ba0c-9f1bce379082 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:08.183431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7b496-7093-47de-ba0c-9f1bce379082 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:08.184631 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72a893-c218-4119-a3cd-bcd0ba0247fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:08.185264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72a893-c218-4119-a3cd-bcd0ba0247fd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:08.185547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72a893-c218-4119-a3cd-bcd0ba0247fd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:08.185825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72a893-c218-4119-a3cd-bcd0ba0247fd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:08.186749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:08.193504 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e72a893-c218-4119-a3cd-bcd0ba0247fd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:08.194945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e72a893-c218-4119-a3cd-bcd0ba0247fd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:08.206684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-0e72a893-c218-4119-a3cd-bcd0ba0247fd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 12:18:08.206684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:08.206684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:08.206684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:08.206684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:08.206684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:08.206684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:08.206684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:08.206684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:08.206684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:08.206684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:08.206684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:08.206684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:08.206684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:08.206684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:08.206684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:08.207719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:08.207719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:08.207719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:08.207719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:08.207719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:08.207719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:08.207719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:08.207719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:08.207719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:08.207719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:08.207719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:08.207719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:08.207719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:08.207719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:08.207719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:08.207719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:08.207719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:08.207719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:08.208651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:08.208651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:08.208651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:08.208651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:08.208651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:08.208651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:08.208651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:08.208651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:08.208651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 12:18:08.208651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:08.208651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 260/514] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:18:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:08.208651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72a893-c218-4119-a3cd-bcd0ba0247fd None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:08.254004 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7b496-7093-47de-ba0c-9f1bce379082 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:08.255462 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7b496-7093-47de-ba0c-9f1bce379082 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:08.283621 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:08.308056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72a893-c218-4119-a3cd-bcd0ba0247fd None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:08.341671 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a67c5-8806-42ce-9f6e-727fdfe59091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:08.342281 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a67c5-8806-42ce-9f6e-727fdfe59091 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:08.342560 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a67c5-8806-42ce-9f6e-727fdfe59091 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:08.342829 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a67c5-8806-42ce-9f6e-727fdfe59091 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:08.353057 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a67c5-8806-42ce-9f6e-727fdfe59091 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:08.354144 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 255/515] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:18:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:18:08.354476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a67c5-8806-42ce-9f6e-727fdfe59091 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:08.354890 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7b496-7093-47de-ba0c-9f1bce379082 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:08.371315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-e70a67c5-8806-42ce-9f6e-727fdfe59091 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:08.371315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:08.371315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:08.371315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:08.371315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:08.371315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:08.371315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:08.371315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:08.371315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:08.371315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:08.371315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:08.371315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:08.371315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:08.371315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:08.371315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:08.372348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:08.372348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:08.372348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:08.372348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:08.372348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:08.372348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:08.372348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:08.372348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:08.372348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:08.372348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:08.372348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:08.372348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:08.372348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:08.372348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:08.372348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:08.372348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:08.372348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:08.373411 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:08.373411 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:08.373411 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:08.373411 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:08.373411 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 261/516] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:08.373411 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a67c5-8806-42ce-9f6e-727fdfe59091 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:08.392892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:08.456629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7b496-7093-47de-ba0c-9f1bce379082 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:08.470894 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a67c5-8806-42ce-9f6e-727fdfe59091 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:08.490454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf0016-8b5a-4bd3-b596-1ca830030c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:08.491326 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf0016-8b5a-4bd3-b596-1ca830030c87 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:08.491608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf0016-8b5a-4bd3-b596-1ca830030c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:08.491881 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf0016-8b5a-4bd3-b596-1ca830030c87 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:08.492622 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf0016-8b5a-4bd3-b596-1ca830030c87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:08.492929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf0016-8b5a-4bd3-b596-1ca830030c87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:08.494339 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf0016-8b5a-4bd3-b596-1ca830030c87 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:08.496924 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:08.505591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0c18f-b027-45b9-8def-c512bbd22e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:08.505757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 256/517] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:08.506205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0c18f-b027-45b9-8def-c512bbd22e45 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:08.506385 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf0016-8b5a-4bd3-b596-1ca830030c87 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:08.506479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0c18f-b027-45b9-8def-c512bbd22e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:08.506748 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0c18f-b027-45b9-8def-c512bbd22e45 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:08.507545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0c18f-b027-45b9-8def-c512bbd22e45 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:08.507850 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0c18f-b027-45b9-8def-c512bbd22e45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:08.509839 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0c18f-b027-45b9-8def-c512bbd22e45 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:08.527384 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 262/518] 10.222.0.20 () {56 vars in 1139 bytes} [Sat Aug 6 12:18:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:08.528180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0c18f-b027-45b9-8def-c512bbd22e45 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:08.598704 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:08.605453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf0016-8b5a-4bd3-b596-1ca830030c87 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:08.627574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0c18f-b027-45b9-8def-c512bbd22e45 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:08.639623 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6e94e-15d5-4eb7-a8e2-15db415a4379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:08.640230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6e94e-15d5-4eb7-a8e2-15db415a4379 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:08.640547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6e94e-15d5-4eb7-a8e2-15db415a4379 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:08.640817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6e94e-15d5-4eb7-a8e2-15db415a4379 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:08.641599 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e6e94e-15d5-4eb7-a8e2-15db415a4379 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:08.641901 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e6e94e-15d5-4eb7-a8e2-15db415a4379 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:08.643332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e6e94e-15d5-4eb7-a8e2-15db415a4379 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:08.660520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda6bbc-3b27-4b85-9029-d31005aedb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:08.660820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 257/519] 10.222.0.20 () {56 vars in 1137 bytes} [Sat Aug 6 12:18:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:08.661223 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda6bbc-3b27-4b85-9029-d31005aedb2c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:08.661408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda6bbc-3b27-4b85-9029-d31005aedb2c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:08.661612 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6e94e-15d5-4eb7-a8e2-15db415a4379 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:08.661715 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda6bbc-3b27-4b85-9029-d31005aedb2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:08.672734 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda6bbc-3b27-4b85-9029-d31005aedb2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:08.674188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda6bbc-3b27-4b85-9029-d31005aedb2c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:08.689592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-3dda6bbc-3b27-4b85-9029-d31005aedb2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:08.689592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:08.689592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:08.689592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:08.689592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:08.689592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:08.689592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:08.689592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:08.689592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:08.689592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:08.689592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:08.689592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:08.689592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:08.689592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:08.689592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:08.690483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:08.690483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:08.690483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:08.690483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:08.690483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:08.690483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:08.690483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:08.690483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:08.690483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:08.690483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:08.690483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:08.690483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:08.690483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:08.690483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:08.690483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:08.690483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:08.690483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:08.691347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:08.691347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:08.691347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:08.691347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:08.691347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 263/520] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:08.691613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda6bbc-3b27-4b85-9029-d31005aedb2c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:08.730366 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:08.762486 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6e94e-15d5-4eb7-a8e2-15db415a4379 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:08.790822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda6bbc-3b27-4b85-9029-d31005aedb2c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:08.795971 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e76f91-05bf-4dfc-9a16-87489a32dce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:08.796618 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e76f91-05bf-4dfc-9a16-87489a32dce1 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:08.796892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e76f91-05bf-4dfc-9a16-87489a32dce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:08.797155 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e76f91-05bf-4dfc-9a16-87489a32dce1 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:08.797920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e76f91-05bf-4dfc-9a16-87489a32dce1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:08.798217 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e76f91-05bf-4dfc-9a16-87489a32dce1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:08.799580 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e76f91-05bf-4dfc-9a16-87489a32dce1 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:08.811450 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 258/521] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:08.812181 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e76f91-05bf-4dfc-9a16-87489a32dce1 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:08.828242 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:08.829344 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebeed8-29ff-4791-9eca-efb641d319b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:08.830033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebeed8-29ff-4791-9eca-efb641d319b7 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:08.830240 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebeed8-29ff-4791-9eca-efb641d319b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:08.830556 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebeed8-29ff-4791-9eca-efb641d319b7 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:08.831316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebeed8-29ff-4791-9eca-efb641d319b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:08.832663 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebeed8-29ff-4791-9eca-efb641d319b7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:08.852438 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 264/522] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Aug 6 12:18:08 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:18:08.853272 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebeed8-29ff-4791-9eca-efb641d319b7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:08.930674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e76f91-05bf-4dfc-9a16-87489a32dce1 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:08.961219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:08.969389 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6d369-8ade-4ba5-9480-7fa33be9ac3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:08.970113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6d369-8ade-4ba5-9480-7fa33be9ac3e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:08.970448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebeed8-29ff-4791-9eca-efb641d319b7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:08.970550 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6d369-8ade-4ba5-9480-7fa33be9ac3e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:08.970828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6d369-8ade-4ba5-9480-7fa33be9ac3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:09.013330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd77ed-9605-4124-83e4-8867325a3713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:09.014942 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd77ed-9605-4124-83e4-8867325a3713 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:09.014942 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd77ed-9605-4124-83e4-8867325a3713 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:09.015155 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd77ed-9605-4124-83e4-8867325a3713 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:09.060151 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6d369-8ade-4ba5-9480-7fa33be9ac3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:09.061636 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6d369-8ade-4ba5-9480-7fa33be9ac3e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:09.077213 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:09.090969 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd77ed-9605-4124-83e4-8867325a3713 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:09.092590 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd77ed-9605-4124-83e4-8867325a3713 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:09.167814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 259/523] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:18:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:18:09.167814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6d369-8ade-4ba5-9480-7fa33be9ac3e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:09.194033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:09.207200 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 265/524] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:18:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:18:09.207200 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd77ed-9605-4124-83e4-8867325a3713 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:09.276671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6d369-8ade-4ba5-9480-7fa33be9ac3e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:09.292657 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:09.315249 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fb0d4-e950-4229-a02b-6d508a7073bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:09.315853 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fb0d4-e950-4229-a02b-6d508a7073bd None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:09.316125 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fb0d4-e950-4229-a02b-6d508a7073bd None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:09.316422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fb0d4-e950-4229-a02b-6d508a7073bd None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:09.316515 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd77ed-9605-4124-83e4-8867325a3713 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:09.317098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319fb0d4-e950-4229-a02b-6d508a7073bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:09.317391 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319fb0d4-e950-4229-a02b-6d508a7073bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:09.319060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319fb0d4-e950-4229-a02b-6d508a7073bd None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:09.330890 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 260/525] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:09.331705 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fb0d4-e950-4229-a02b-6d508a7073bd None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:09.355190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e3da3-d575-428a-ac69-c7f234a14110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:09.355830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e3da3-d575-428a-ac69-c7f234a14110 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:09.356106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e3da3-d575-428a-ac69-c7f234a14110 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:09.356405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e3da3-d575-428a-ac69-c7f234a14110 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:09.363153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e3da3-d575-428a-ac69-c7f234a14110 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:09.364636 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e3da3-d575-428a-ac69-c7f234a14110 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:09.377463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-e44e3da3-d575-428a-ac69-c7f234a14110 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:09.377463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:09.377463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:09.377463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:09.377463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:09.377463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:09.377463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:09.377463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:09.377463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:09.377463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:09.377463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:09.377463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:09.377463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:09.377463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:09.377463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:09.378322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:09.378322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:09.378322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:09.378322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:09.378322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:09.378322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:09.378322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:09.378322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:09.378322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:09.378322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:09.378322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:09.378322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:09.378322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:09.378322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:09.378322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:09.378322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:09.378322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:09.379146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:09.379146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:09.379146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:09.379146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:09.379146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:09.379146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:09.379146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:09.379146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:09.379146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 266/526] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:09.379631 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e3da3-d575-428a-ac69-c7f234a14110 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:09.437163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:09.445521 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fb0d4-e950-4229-a02b-6d508a7073bd None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:09.481828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11812999-195a-4531-b668-b5d0d7c8b904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:09.482459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11812999-195a-4531-b668-b5d0d7c8b904 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:09.482749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11812999-195a-4531-b668-b5d0d7c8b904 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:09.483006 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11812999-195a-4531-b668-b5d0d7c8b904 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:09.483278 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e3da3-d575-428a-ac69-c7f234a14110 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:09.494340 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11812999-195a-4531-b668-b5d0d7c8b904 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:09.495783 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11812999-195a-4531-b668-b5d0d7c8b904 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:09.510410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-11812999-195a-4531-b668-b5d0d7c8b904 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:09.510410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:09.510410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:09.510410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:09.510410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:09.510410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:09.510410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:09.510410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:09.510410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:09.510410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:09.510410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:09.510410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:09.510410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:09.510410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:09.510410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:09.511264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:09.511264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:09.511264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:09.511264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:09.511264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:09.511264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:09.511264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:09.511264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:09.511264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:09.511264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:09.511264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:09.511264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:09.511264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:09.511264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:09.511264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:09.511264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:09.511264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:09.512106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:09.512106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:09.512106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:09.512106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:09.512106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 261/527] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:09.512428 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11812999-195a-4531-b668-b5d0d7c8b904 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:09.526659 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a4a92-33a3-4503-9bd0-5e2356ab7519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:09.527700 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a4a92-33a3-4503-9bd0-5e2356ab7519 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:09.527922 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a4a92-33a3-4503-9bd0-5e2356ab7519 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:09.528193 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a4a92-33a3-4503-9bd0-5e2356ab7519 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:09.539812 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a4a92-33a3-4503-9bd0-5e2356ab7519 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:09.541347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a4a92-33a3-4503-9bd0-5e2356ab7519 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:09.545289 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:09.556529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-5c1a4a92-33a3-4503-9bd0-5e2356ab7519 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:09.556529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:09.556529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:09.556529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:09.556529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:09.556529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:09.556529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:09.556529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:09.556529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:09.556529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:09.556529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:09.556529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:09.556529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:09.556529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:09.556529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:09.557401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:09.557401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:09.557401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:09.557401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:09.557401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:09.557401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:09.557401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:09.557401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:09.557401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:09.557401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:09.557401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:09.557401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:09.557401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:09.557401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:09.557401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:09.557401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:09.557401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:09.558330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:09.558330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:09.558330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:09.558330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:09.558330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 267/528] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:09.561178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a4a92-33a3-4503-9bd0-5e2356ab7519 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:09.628394 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11812999-195a-4531-b668-b5d0d7c8b904 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:09.662785 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01bc1cf-ea6e-486d-be59-bc37b7e4d3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:09.663424 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01bc1cf-ea6e-486d-be59-bc37b7e4d3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:09.663680 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01bc1cf-ea6e-486d-be59-bc37b7e4d3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:09.663936 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01bc1cf-ea6e-486d-be59-bc37b7e4d3e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:09.667102 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a4a92-33a3-4503-9bd0-5e2356ab7519 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:09.670788 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01bc1cf-ea6e-486d-be59-bc37b7e4d3e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:09.672337 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01bc1cf-ea6e-486d-be59-bc37b7e4d3e0 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:09.674228 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:09.689469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-c01bc1cf-ea6e-486d-be59-bc37b7e4d3e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:09.689469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:09.689469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:09.689469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:09.689469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:09.689469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:09.689469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:09.689469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:09.689469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:09.689469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:09.689469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:09.689469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:09.689469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:09.689469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:09.689469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:09.690331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:09.690331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:09.690331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:09.690331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:09.690331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:09.690331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:09.690331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:09.690331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:09.690331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:09.690331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:09.690331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:09.690331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:09.690331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:09.690331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:09.690331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:09.690331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:09.690331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:09.696473 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:09.696473 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:09.696473 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:09.696473 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:09.696473 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:09.696473 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:09.696473 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:09.696473 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:09.696473 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 262/529] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:09.696473 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01bc1cf-ea6e-486d-be59-bc37b7e4d3e0 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:09.706852 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691d92b-9878-441c-8ec1-a834263f9fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:09.707666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691d92b-9878-441c-8ec1-a834263f9fbf None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:09.707943 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691d92b-9878-441c-8ec1-a834263f9fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:09.708228 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4691d92b-9878-441c-8ec1-a834263f9fbf None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:09.709006 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691d92b-9878-441c-8ec1-a834263f9fbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:09.709314 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691d92b-9878-441c-8ec1-a834263f9fbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:09.710730 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4691d92b-9878-441c-8ec1-a834263f9fbf None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:09.733580 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 268/530] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:09.733580 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691d92b-9878-441c-8ec1-a834263f9fbf None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:09.786948 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:09.804906 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01bc1cf-ea6e-486d-be59-bc37b7e4d3e0 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:09.837894 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4691d92b-9878-441c-8ec1-a834263f9fbf None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:09.838544 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b9f3c-dbbc-4dac-ad8c-88ce59daa96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:09.839150 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b9f3c-dbbc-4dac-ad8c-88ce59daa96e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:09.839427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b9f3c-dbbc-4dac-ad8c-88ce59daa96e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:09.839697 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b9f3c-dbbc-4dac-ad8c-88ce59daa96e None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:09.840417 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b9f3c-dbbc-4dac-ad8c-88ce59daa96e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:09.840718 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b9f3c-dbbc-4dac-ad8c-88ce59daa96e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:09.842075 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b9f3c-dbbc-4dac-ad8c-88ce59daa96e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:09.854273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 263/531] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:09.855066 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b9f3c-dbbc-4dac-ad8c-88ce59daa96e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:09.883117 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8aec5-c1ef-4320-8f7f-96169981fbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:09.884600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8aec5-c1ef-4320-8f7f-96169981fbb2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:09.885023 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8aec5-c1ef-4320-8f7f-96169981fbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:09.885298 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8aec5-c1ef-4320-8f7f-96169981fbb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:09.885988 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8aec5-c1ef-4320-8f7f-96169981fbb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:09.886286 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8aec5-c1ef-4320-8f7f-96169981fbb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:09.887698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8aec5-c1ef-4320-8f7f-96169981fbb2 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:09.903997 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 269/532] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:09.905196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8aec5-c1ef-4320-8f7f-96169981fbb2 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:09.934442 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:09.962848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b9f3c-dbbc-4dac-ad8c-88ce59daa96e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:10.007561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e25df-8982-4ba3-a103-736929887210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:10.008328 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e25df-8982-4ba3-a103-736929887210 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:10.008771 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e25df-8982-4ba3-a103-736929887210 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:10.009676 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e25df-8982-4ba3-a103-736929887210 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:10.010546 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e25df-8982-4ba3-a103-736929887210 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:10.010951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e25df-8982-4ba3-a103-736929887210 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:10.012516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e25df-8982-4ba3-a103-736929887210 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:10.015025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8aec5-c1ef-4320-8f7f-96169981fbb2 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:10.032010 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 264/533] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:10.037265 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e25df-8982-4ba3-a103-736929887210 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:10.046666 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:10.053893 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b55cf-057e-434c-ab4a-cd5a17a15110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:10.054521 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b55cf-057e-434c-ab4a-cd5a17a15110 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:10.054796 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b55cf-057e-434c-ab4a-cd5a17a15110 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:10.055067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b55cf-057e-434c-ab4a-cd5a17a15110 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:10.055885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b55cf-057e-434c-ab4a-cd5a17a15110 None admin] RBAC: Adding query filter params (role.id=785cca921bad4984a86e45ce6c5601c2, scope.project.id=13fba33c46cf40bbb15a08c690b4c98c, user.id=7610d6289efb44599d0cee03b93f9938) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:10.056261 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b55cf-057e-434c-ab4a-cd5a17a15110 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:10.057725 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b55cf-057e-434c-ab4a-cd5a17a15110 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:10.072339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 270/534] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:18:09 2022] GET /identity/v3/role_assignments?user.id=7610d6289efb44599d0cee03b93f9938&scope.project.id=13fba33c46cf40bbb15a08c690b4c98c&role.id=785cca921bad4984a86e45ce6c5601c2 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:10.072339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b55cf-057e-434c-ab4a-cd5a17a15110 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:10.158698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e25df-8982-4ba3-a103-736929887210 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:10.189753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:10.200151 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef2fd5-116f-4679-8e16-c46adb79c89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:10.200670 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b55cf-057e-434c-ab4a-cd5a17a15110 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:10.201043 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef2fd5-116f-4679-8e16-c46adb79c89d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:10.201400 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef2fd5-116f-4679-8e16-c46adb79c89d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:10.201745 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef2fd5-116f-4679-8e16-c46adb79c89d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:10.209628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef2fd5-116f-4679-8e16-c46adb79c89d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:10.211521 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef2fd5-116f-4679-8e16-c46adb79c89d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:10.225534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-90ef2fd5-116f-4679-8e16-c46adb79c89d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:10.225534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:10.225534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:10.225534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:10.225534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:10.225534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:10.225534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:10.225534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:10.225534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:10.225534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:10.225534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:10.225534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:10.225534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:10.225534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:10.225534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:10.226441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:10.226441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:10.226441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:10.226441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:10.226441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:10.226441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:10.226441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:10.226441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:10.226441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:10.226441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:10.226441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:10.226441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:10.226441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:10.226441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:10.226441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:10.226441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:10.226441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:10.227281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:10.227281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:10.227281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:10.227281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:10.227281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:10.227281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:10.227281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:10.227281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:10.231892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 265/535] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:10.233044 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef2fd5-116f-4679-8e16-c46adb79c89d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:10.240893 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5742b96-f8ab-4c76-bf2e-9dd35837c17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:10.242165 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5742b96-f8ab-4c76-bf2e-9dd35837c17f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:10.242651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5742b96-f8ab-4c76-bf2e-9dd35837c17f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:10.243247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5742b96-f8ab-4c76-bf2e-9dd35837c17f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:10.253210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5742b96-f8ab-4c76-bf2e-9dd35837c17f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:10.254951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5742b96-f8ab-4c76-bf2e-9dd35837c17f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:10.266780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-b5742b96-f8ab-4c76-bf2e-9dd35837c17f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:10.266780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:10.266780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:10.266780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:10.266780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:10.266780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:10.266780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:10.266780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:10.266780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:10.266780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:10.266780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:10.266780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:10.266780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:10.266780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:10.266780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:10.267632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:10.267632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:10.267632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:10.267632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:10.267632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:10.267632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:10.267632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:10.267632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:10.267632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:10.267632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:10.267632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:10.267632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:10.267632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:10.267632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:10.267632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:10.267632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:10.267632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:10.268856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:10.268856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:10.268856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:10.268856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:10.268856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:10.268856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:10.268856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:10.268856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:10.268856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 271/536] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:10.269458 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5742b96-f8ab-4c76-bf2e-9dd35837c17f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:10.311685 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:10.356137 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef2fd5-116f-4679-8e16-c46adb79c89d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:10.380246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5742b96-f8ab-4c76-bf2e-9dd35837c17f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:10.395680 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed6f86-df4a-4103-b1b7-e5a5578be669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:10.396549 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed6f86-df4a-4103-b1b7-e5a5578be669 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:10.396995 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed6f86-df4a-4103-b1b7-e5a5578be669 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:10.397406 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed6f86-df4a-4103-b1b7-e5a5578be669 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:10.398452 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed6f86-df4a-4103-b1b7-e5a5578be669 None admin] RBAC: Adding query filter params (role.id=785cca921bad4984a86e45ce6c5601c2, scope.project.id=13fba33c46cf40bbb15a08c690b4c98c, user.id=524b728bf63c4ca88e0f25e531cf5056) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:10.398992 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed6f86-df4a-4103-b1b7-e5a5578be669 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:10.401113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed6f86-df4a-4103-b1b7-e5a5578be669 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:10.407614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 266/537] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:18:10 2022] GET /identity/v3/role_assignments?user.id=524b728bf63c4ca88e0f25e531cf5056&scope.project.id=13fba33c46cf40bbb15a08c690b4c98c&role.id=785cca921bad4984a86e45ce6c5601c2 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:10.408936 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed6f86-df4a-4103-b1b7-e5a5578be669 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:10.418230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3be129-1840-46db-bc9b-7b0b5f59644d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:10.418858 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3be129-1840-46db-bc9b-7b0b5f59644d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:10.419138 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3be129-1840-46db-bc9b-7b0b5f59644d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:10.419413 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3be129-1840-46db-bc9b-7b0b5f59644d None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:10.420174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3be129-1840-46db-bc9b-7b0b5f59644d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:10.420528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3be129-1840-46db-bc9b-7b0b5f59644d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:10.422146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3be129-1840-46db-bc9b-7b0b5f59644d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:10.437781 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 272/538] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:10.445417 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3be129-1840-46db-bc9b-7b0b5f59644d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:10.466951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:10.563903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3be129-1840-46db-bc9b-7b0b5f59644d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:10.607217 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1b002-1c8c-48d9-898e-2bf8f0ca3393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:10.607844 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1b002-1c8c-48d9-898e-2bf8f0ca3393 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:10.608121 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1b002-1c8c-48d9-898e-2bf8f0ca3393 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:10.608460 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1b002-1c8c-48d9-898e-2bf8f0ca3393 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:10.609258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f1b002-1c8c-48d9-898e-2bf8f0ca3393 None admin] RBAC: Adding query filter params (role.id=785cca921bad4984a86e45ce6c5601c2, scope.project.id=13fba33c46cf40bbb15a08c690b4c98c, user.id=55563aaa662246c5aaf107145475bd76) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:10.609569 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f1b002-1c8c-48d9-898e-2bf8f0ca3393 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:10.611280 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f1b002-1c8c-48d9-898e-2bf8f0ca3393 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:10.617943 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 273/539] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:18:10 2022] GET /identity/v3/role_assignments?user.id=55563aaa662246c5aaf107145475bd76&scope.project.id=13fba33c46cf40bbb15a08c690b4c98c&role.id=785cca921bad4984a86e45ce6c5601c2 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:10.647856 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed6f86-df4a-4103-b1b7-e5a5578be669 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:10.694654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a36d49-24ea-4bfc-a0d1-d31d570e6daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:10.694654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:10.694654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a36d49-24ea-4bfc-a0d1-d31d570e6daf None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:10.694654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a36d49-24ea-4bfc-a0d1-d31d570e6daf None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:10.695479 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a36d49-24ea-4bfc-a0d1-d31d570e6daf None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:10.695479 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a36d49-24ea-4bfc-a0d1-d31d570e6daf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:10.695479 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a36d49-24ea-4bfc-a0d1-d31d570e6daf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:10.697562 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a36d49-24ea-4bfc-a0d1-d31d570e6daf None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:10.720071 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 267/540] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:10.725921 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1b002-1c8c-48d9-898e-2bf8f0ca3393 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:10.838736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:10.876574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1b002-1c8c-48d9-898e-2bf8f0ca3393 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:10.913682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d39710-e6f7-4283-9d31-1165271cd6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:10.914300 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d39710-e6f7-4283-9d31-1165271cd6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:10.914579 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d39710-e6f7-4283-9d31-1165271cd6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:10.914850 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d39710-e6f7-4283-9d31-1165271cd6b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:10.915681 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d39710-e6f7-4283-9d31-1165271cd6b3 None admin] RBAC: Adding query filter params (role.id=785cca921bad4984a86e45ce6c5601c2, scope.project.id=13fba33c46cf40bbb15a08c690b4c98c, user.id=d7a3cbb9c6a34fe2b280873771532e4b) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:10.915989 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d39710-e6f7-4283-9d31-1165271cd6b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:10.917499 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d39710-e6f7-4283-9d31-1165271cd6b3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:10.924326 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 274/541] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:18:10 2022] GET /identity/v3/role_assignments?user.id=d7a3cbb9c6a34fe2b280873771532e4b&scope.project.id=13fba33c46cf40bbb15a08c690b4c98c&role.id=785cca921bad4984a86e45ce6c5601c2 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:10.953262 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5855ed-d677-4fce-9c7a-53fa378109a6 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:10.953577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5855ed-d677-4fce-9c7a-53fa378109a6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:10.953821 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5855ed-d677-4fce-9c7a-53fa378109a6 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:10.955299 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 268/542] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:10.961363 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a222b-015a-48be-8e95-c016ddcb2407 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:10.961981 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a222b-015a-48be-8e95-c016ddcb2407 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:10.962987 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a222b-015a-48be-8e95-c016ddcb2407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:10.983056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-7c5a222b-015a-48be-8e95-c016ddcb2407 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:10.984712 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7c5a222b-015a-48be-8e95-c016ddcb2407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:11.090301 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 275/543] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:18:11.114057 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5855ed-d677-4fce-9c7a-53fa378109a6 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:11.231226 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5855ed-d677-4fce-9c7a-53fa378109a6 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:11.273974 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd1cdb-1463-49eb-b4fa-b8a27c6eaf43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:11.274955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd1cdb-1463-49eb-b4fa-b8a27c6eaf43 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:11.275404 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd1cdb-1463-49eb-b4fa-b8a27c6eaf43 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:11.275857 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd1cdb-1463-49eb-b4fa-b8a27c6eaf43 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:11.290988 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd1cdb-1463-49eb-b4fa-b8a27c6eaf43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:11.296352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-0cbd1cdb-1463-49eb-b4fa-b8a27c6eaf43 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:11.296352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:11.296352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:11.296352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:11.296352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:11.296352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:11.296352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:11.296352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:11.296352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:11.296352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:11.296352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:11.296352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:11.296352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:11.296352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:11.296352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:11.296352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:11.297231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:11.297231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:11.297231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:11.297231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:11.297231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:11.297231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:11.297231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:11.297231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:11.297231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:11.297231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:11.297231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:11.297231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:11.297231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:11.297231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:11.297231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:11.297231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:11.298026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:11.298026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:11.298026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:11.298026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:11.298026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:11.298026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:11.298026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:11.298026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:11.298026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:11.298026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:11.298026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:11.298026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:11.299973 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 269/544] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Aug 6 12:18:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:11.306008 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a222b-015a-48be-8e95-c016ddcb2407 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:11.387496 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:11.443992 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a222b-015a-48be-8e95-c016ddcb2407 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:11.494771 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d5193-1e3c-47bc-b415-3daa5cd3904f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:11.495657 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d5193-1e3c-47bc-b415-3daa5cd3904f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:11.496074 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d5193-1e3c-47bc-b415-3daa5cd3904f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:11.496518 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d5193-1e3c-47bc-b415-3daa5cd3904f None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:11.497371 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34d5193-1e3c-47bc-b415-3daa5cd3904f None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:11.497831 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34d5193-1e3c-47bc-b415-3daa5cd3904f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:11.499349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34d5193-1e3c-47bc-b415-3daa5cd3904f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:11.512907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 276/545] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Aug 6 12:18:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:11.518101 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd1cdb-1463-49eb-b4fa-b8a27c6eaf43 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:11.582668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:11.640330 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd1cdb-1463-49eb-b4fa-b8a27c6eaf43 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:11.678760 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae43ce-c351-4088-b731-0b2ef3dd1f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:11.679620 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae43ce-c351-4088-b731-0b2ef3dd1f00 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:11.680036 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae43ce-c351-4088-b731-0b2ef3dd1f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:11.680478 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae43ce-c351-4088-b731-0b2ef3dd1f00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:11.694066 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae43ce-c351-4088-b731-0b2ef3dd1f00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:11.695703 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae43ce-c351-4088-b731-0b2ef3dd1f00 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:11.717877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-78ae43ce-c351-4088-b731-0b2ef3dd1f00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:11.717877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:11.717877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:11.717877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:11.717877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:11.717877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:11.717877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:11.717877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:11.717877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:11.717877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:11.717877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:11.717877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:11.717877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:11.717877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:11.717877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:11.718919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:11.718919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:11.718919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:11.718919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:11.718919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:11.718919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:11.718919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:11.718919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:11.718919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:11.718919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:11.718919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:11.718919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:11.718919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:11.718919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:11.718919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:11.718919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:11.718919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:11.719959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:11.719959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:11.719959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:11.719959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:11.721439 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 270/546] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:11.727117 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d5193-1e3c-47bc-b415-3daa5cd3904f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:11.849157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d5193-1e3c-47bc-b415-3daa5cd3904f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:11.878411 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:11.885357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6dfe15-260c-44db-b72a-47a5610459ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:11.885975 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6dfe15-260c-44db-b72a-47a5610459ed None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:11.886252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6dfe15-260c-44db-b72a-47a5610459ed None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:11.886521 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6dfe15-260c-44db-b72a-47a5610459ed None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:11.887201 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6dfe15-260c-44db-b72a-47a5610459ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:11.887495 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6dfe15-260c-44db-b72a-47a5610459ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:11.888950 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6dfe15-260c-44db-b72a-47a5610459ed None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:11.901812 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 277/547] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:11.909323 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae43ce-c351-4088-b731-0b2ef3dd1f00 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:12.053321 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae43ce-c351-4088-b731-0b2ef3dd1f00 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:12.055242 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:12.092814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3ffba-b2f3-4be6-94ac-f42824c1e3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:12.093812 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3ffba-b2f3-4be6-94ac-f42824c1e3f7 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:12.094290 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3ffba-b2f3-4be6-94ac-f42824c1e3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:12.094759 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3ffba-b2f3-4be6-94ac-f42824c1e3f7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:12.103279 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3ffba-b2f3-4be6-94ac-f42824c1e3f7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:12.104992 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3ffba-b2f3-4be6-94ac-f42824c1e3f7 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:12.116834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-2fa3ffba-b2f3-4be6-94ac-f42824c1e3f7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 12:18:12.116834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:12.116834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:12.116834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:12.116834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:12.116834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:12.116834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:12.116834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:12.116834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:12.116834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:12.116834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:12.116834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:12.116834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:12.116834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:12.116834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:12.116834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:12.117701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:12.117701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:12.117701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:12.117701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:12.117701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:12.117701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:12.117701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:12.117701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:12.117701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:12.117701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:12.117701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:12.117701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:12.117701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:12.117701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:12.117701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:12.117701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:12.117701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:12.117701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:12.119744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:12.119744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:12.119744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:12.119744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:12.119744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:12.119744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:12.119744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:12.119744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:12.119744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 12:18:12.119744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:12.122542 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 271/548] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:18:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:12.128211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6dfe15-260c-44db-b72a-47a5610459ed None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:12.233849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6dfe15-260c-44db-b72a-47a5610459ed None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:12.270528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d1909-f6e0-4707-b11b-900a0883bdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:12.271497 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d1909-f6e0-4707-b11b-900a0883bdf7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:12.271976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d1909-f6e0-4707-b11b-900a0883bdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:12.272520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d1909-f6e0-4707-b11b-900a0883bdf7 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:12.273510 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d1909-f6e0-4707-b11b-900a0883bdf7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:12.274007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d1909-f6e0-4707-b11b-900a0883bdf7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:12.275814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d1909-f6e0-4707-b11b-900a0883bdf7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:12.295194 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 278/549] 10.222.0.20 () {56 vars in 1135 bytes} [Sat Aug 6 12:18:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:12.301037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3ffba-b2f3-4be6-94ac-f42824c1e3f7 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:12.424279 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3ffba-b2f3-4be6-94ac-f42824c1e3f7 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:12.458638 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c9451-8e5d-4cb3-ad37-fa085d982c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:12.459573 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c9451-8e5d-4cb3-ad37-fa085d982c14 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:12.460051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c9451-8e5d-4cb3-ad37-fa085d982c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:12.460573 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c9451-8e5d-4cb3-ad37-fa085d982c14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:12.490994 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:12.541824 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c9451-8e5d-4cb3-ad37-fa085d982c14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:12.543286 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c9451-8e5d-4cb3-ad37-fa085d982c14 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:12.672461 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 272/550] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:18:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:18:12.674395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d1909-f6e0-4707-b11b-900a0883bdf7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:12.755591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:12.818049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d1909-f6e0-4707-b11b-900a0883bdf7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:12.855214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32905b24-dd0d-4a4e-8fac-eb6b374d3968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:12.855853 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32905b24-dd0d-4a4e-8fac-eb6b374d3968 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:12.856135 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32905b24-dd0d-4a4e-8fac-eb6b374d3968 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:12.856440 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32905b24-dd0d-4a4e-8fac-eb6b374d3968 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:12.867470 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32905b24-dd0d-4a4e-8fac-eb6b374d3968 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:12.869058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32905b24-dd0d-4a4e-8fac-eb6b374d3968 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:12.873762 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda167f-8a83-40ca-a20e-3c2639133bfa None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:12.874213 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda167f-8a83-40ca-a20e-3c2639133bfa None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:12.874311 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda167f-8a83-40ca-a20e-3c2639133bfa None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:12.876794 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 273/551] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:12.885575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-32905b24-dd0d-4a4e-8fac-eb6b374d3968 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:12.885575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:12.885575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:12.885575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:12.885575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:12.885575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:12.885575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:12.885575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:12.885575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:12.885575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:12.885575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:12.885575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:12.885575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:12.885575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:12.885575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:12.886522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:12.886522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:12.886522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:12.886522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:12.886522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:12.886522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:12.886522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:12.886522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:12.886522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:12.886522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:12.886522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:12.886522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:12.886522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:12.886522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:12.886522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:12.886522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:12.886522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:12.887489 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:12.887489 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:12.887489 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:12.887489 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:12.887708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525632a1-6a60-4d5f-bbef-1d86a11b48e4 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:12.887708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525632a1-6a60-4d5f-bbef-1d86a11b48e4 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:12.887708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525632a1-6a60-4d5f-bbef-1d86a11b48e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:12.887892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 279/552] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:12.892340 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32905b24-dd0d-4a4e-8fac-eb6b374d3968 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:12.914194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-525632a1-6a60-4d5f-bbef-1d86a11b48e4 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:12.916111 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-525632a1-6a60-4d5f-bbef-1d86a11b48e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:13.016950 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:13.022893 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32905b24-dd0d-4a4e-8fac-eb6b374d3968 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:13.027649 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:13.047332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 274/553] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:18:13.048991 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132fb7f-e9e2-4215-8f95-add2ce0cb6ce None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:13.049472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132fb7f-e9e2-4215-8f95-add2ce0cb6ce None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:13.049936 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132fb7f-e9e2-4215-8f95-add2ce0cb6ce None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:13.055434 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 275/554] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:13.057591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fed632-0efb-4ed8-a968-d089cc1b0f49 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:13.058293 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fed632-0efb-4ed8-a968-d089cc1b0f49 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:13.058764 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fed632-0efb-4ed8-a968-d089cc1b0f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:13.079977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-a4fed632-0efb-4ed8-a968-d089cc1b0f49 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:13.082230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a4fed632-0efb-4ed8-a968-d089cc1b0f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:13.089244 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ac029-c34e-4607-856d-4e61771077a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:13.089864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ac029-c34e-4607-856d-4e61771077a7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:13.090144 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ac029-c34e-4607-856d-4e61771077a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:13.090422 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ac029-c34e-4607-856d-4e61771077a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:13.091115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ac029-c34e-4607-856d-4e61771077a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:13.091416 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ac029-c34e-4607-856d-4e61771077a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:13.092934 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ac029-c34e-4607-856d-4e61771077a7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:13.106508 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 280/555] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:13.111608 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ac029-c34e-4607-856d-4e61771077a7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:13.173127 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 276/556] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:18:13.174400 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fed632-0efb-4ed8-a968-d089cc1b0f49 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:13.248498 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ac029-c34e-4607-856d-4e61771077a7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:13.306923 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fed632-0efb-4ed8-a968-d089cc1b0f49 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:13.320748 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00adfcda-3435-41ad-a9b2-e291814a3fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:13.321517 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00adfcda-3435-41ad-a9b2-e291814a3fff None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:13.321808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00adfcda-3435-41ad-a9b2-e291814a3fff None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:13.322086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00adfcda-3435-41ad-a9b2-e291814a3fff None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:13.335011 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:13.339089 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00adfcda-3435-41ad-a9b2-e291814a3fff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:13.344250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-00adfcda-3435-41ad-a9b2-e291814a3fff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:13.344250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:13.344250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:13.344250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:13.344250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:13.344250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:13.344250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:13.344250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:13.344250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:13.344250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:13.344250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:13.344250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:13.344250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:13.344250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:13.344250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:13.344250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:13.345396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:13.345396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:13.345396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:13.345396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:13.345396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:13.345396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:13.345396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:13.345396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:13.345396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:13.345396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:13.345396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:13.345396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:13.345396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:13.345396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:13.345396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:13.345396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:13.346403 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:13.346403 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:13.346403 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:13.346403 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:13.346403 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:13.346403 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:13.346403 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:13.346403 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:13.346403 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:13.346403 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:13.346403 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:13.346403 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:13.352989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd24a6-1c28-45b1-9e16-5c81975e8983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:13.352989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd24a6-1c28-45b1-9e16-5c81975e8983 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:13.352989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd24a6-1c28-45b1-9e16-5c81975e8983 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:13.353286 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd24a6-1c28-45b1-9e16-5c81975e8983 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:13.353672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 281/557] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Aug 6 12:18:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:13.354681 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00adfcda-3435-41ad-a9b2-e291814a3fff None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:13.362451 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd24a6-1c28-45b1-9e16-5c81975e8983 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:13.364069 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd24a6-1c28-45b1-9e16-5c81975e8983 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:13.380194 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:13.381960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-d8bd24a6-1c28-45b1-9e16-5c81975e8983 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:13.381960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:13.381960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:13.381960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:13.381960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:13.381960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:13.381960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:13.381960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:13.381960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:13.381960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:13.381960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:13.381960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:13.381960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:13.381960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:13.381960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:13.383570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:13.383570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:13.383570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:13.383570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:13.383570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:13.383570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:13.383570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:13.383570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:13.383570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:13.383570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:13.383570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:13.383570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:13.383570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:13.383570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:13.383570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:13.383570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:13.383570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:13.384783 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:13.384783 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:13.384783 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:13.384783 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:13.384783 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:13.384783 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:13.384783 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:13.384783 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:13.390679 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 277/558] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:13.390679 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd24a6-1c28-45b1-9e16-5c81975e8983 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:13.476921 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00adfcda-3435-41ad-a9b2-e291814a3fff None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:13.505223 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd24a6-1c28-45b1-9e16-5c81975e8983 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:13.513026 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7176345-8ea8-4552-a31d-e842e1037c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:13.515541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7176345-8ea8-4552-a31d-e842e1037c4a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:13.515541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7176345-8ea8-4552-a31d-e842e1037c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:13.515541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7176345-8ea8-4552-a31d-e842e1037c4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:13.525231 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7176345-8ea8-4552-a31d-e842e1037c4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:13.530964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-d7176345-8ea8-4552-a31d-e842e1037c4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:13.530964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:13.530964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:13.530964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:13.530964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:13.530964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:13.530964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:13.530964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:13.530964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:13.530964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:13.530964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:13.530964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:13.530964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:13.530964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:13.530964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:13.530964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:13.532081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:13.532081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:13.532081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:13.532081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:13.532081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:13.532081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:13.532081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:13.532081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:13.532081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:13.532081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:13.532081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:13.532081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:13.532081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:13.532081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:13.532081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:13.532081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:13.532999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:13.532999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:13.532999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:13.532999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:13.532999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:13.532999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:13.532999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:13.532999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:13.532999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:13.532999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:13.532999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:13.532999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:13.532999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 282/559] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Aug 6 12:18:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:13.533765 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c9d5b-ae61-4c17-bbec-442a07ead9ec None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:13.533765 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c9d5b-ae61-4c17-bbec-442a07ead9ec None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:13.533884 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c9d5b-ae61-4c17-bbec-442a07ead9ec None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:13.535761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 283/560] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:13.538027 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c9d5b-ae61-4c17-bbec-442a07ead9ec None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:13.545887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c04776-d32b-445f-ace7-2c5df7f6c2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:13.546512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c04776-d32b-445f-ace7-2c5df7f6c2ee None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:13.546791 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c04776-d32b-445f-ace7-2c5df7f6c2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:13.547103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c04776-d32b-445f-ace7-2c5df7f6c2ee None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:13.547828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c04776-d32b-445f-ace7-2c5df7f6c2ee None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:13.548132 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c04776-d32b-445f-ace7-2c5df7f6c2ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:13.549553 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c04776-d32b-445f-ace7-2c5df7f6c2ee None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:13.559397 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 278/561] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Aug 6 12:18:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:13.560731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ae6be-c991-4399-aad8-0cf4bb825fca None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:13.561012 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ae6be-c991-4399-aad8-0cf4bb825fca None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:13.561279 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ae6be-c991-4399-aad8-0cf4bb825fca None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:13.562620 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 279/562] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:13.567530 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ae6be-c991-4399-aad8-0cf4bb825fca None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:13.618381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:13.632810 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:13.660623 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c9d5b-ae61-4c17-bbec-442a07ead9ec None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:13.678659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ae6be-c991-4399-aad8-0cf4bb825fca None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:13.695401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d05a9-d8e4-4362-a665-6400289293e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:13.696018 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d05a9-d8e4-4362-a665-6400289293e3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:13.696339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d05a9-d8e4-4362-a665-6400289293e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:13.696621 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d05a9-d8e4-4362-a665-6400289293e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:13.697362 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d05a9-d8e4-4362-a665-6400289293e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:13.697662 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d05a9-d8e4-4362-a665-6400289293e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:13.699078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d05a9-d8e4-4362-a665-6400289293e3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:13.710899 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 284/563] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:13.711892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03a58d-e294-4180-9f8f-bc1e56903c5a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:13.712197 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03a58d-e294-4180-9f8f-bc1e56903c5a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:13.712294 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f6f34-1102-4f7e-96ef-3462984bfb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:13.712524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03a58d-e294-4180-9f8f-bc1e56903c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:13.712898 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f6f34-1102-4f7e-96ef-3462984bfb53 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:13.713177 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f6f34-1102-4f7e-96ef-3462984bfb53 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:13.713446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f6f34-1102-4f7e-96ef-3462984bfb53 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:13.714146 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f6f34-1102-4f7e-96ef-3462984bfb53 None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:13.714461 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f6f34-1102-4f7e-96ef-3462984bfb53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:13.715790 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f6f34-1102-4f7e-96ef-3462984bfb53 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:13.723910 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 280/564] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Aug 6 12:18:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:13.724539 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f6f34-1102-4f7e-96ef-3462984bfb53 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:13.730264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0b03a58d-e294-4180-9f8f-bc1e56903c5a None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:13.731461 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0b03a58d-e294-4180-9f8f-bc1e56903c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:13.796922 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 285/565] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:18:13.798158 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c3e3d-35a7-4b5f-8827-1d9a50bd9ecf None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:13.798447 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c3e3d-35a7-4b5f-8827-1d9a50bd9ecf None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:13.798727 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c3e3d-35a7-4b5f-8827-1d9a50bd9ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:13.815816 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e33c3e3d-35a7-4b5f-8827-1d9a50bd9ecf None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:13.816961 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e33c3e3d-35a7-4b5f-8827-1d9a50bd9ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:13.836118 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f6f34-1102-4f7e-96ef-3462984bfb53 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:13.843151 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:13.872219 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb659121-2308-4377-ab03-727bf47fb7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:13.872887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb659121-2308-4377-ab03-727bf47fb7ee None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:13.873164 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb659121-2308-4377-ab03-727bf47fb7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:13.873434 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb659121-2308-4377-ab03-727bf47fb7ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:13.880358 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:13.883710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb659121-2308-4377-ab03-727bf47fb7ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:13.893729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 286/566] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:18:13.893879 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb659121-2308-4377-ab03-727bf47fb7ee None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:13.893998 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c3e3d-35a7-4b5f-8827-1d9a50bd9ecf None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:13.907561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-fb659121-2308-4377-ab03-727bf47fb7ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:13.907561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:13.907561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:13.907561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:13.907561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:13.907561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:13.907561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:13.907561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:13.907561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:13.907561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:13.907561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:13.907561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:13.907561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:13.907561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:13.907561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:13.908616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:13.908616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:13.908616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:13.908616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:13.908616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:13.908616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:13.908616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:13.908616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:13.908616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:13.908616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:13.908616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:13.908616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:13.908616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:13.908616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:13.908616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:13.908616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:13.908616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:13.909674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:13.909674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:13.909674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:13.909674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:13.909674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 281/567] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:13.909674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb659121-2308-4377-ab03-727bf47fb7ee None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:13.988665 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c3e3d-35a7-4b5f-8827-1d9a50bd9ecf None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:14.010360 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb659121-2308-4377-ab03-727bf47fb7ee None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:14.027129 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e497aeb-6bbc-4c3c-908f-602edcfc1818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:14.027738 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e497aeb-6bbc-4c3c-908f-602edcfc1818 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:14.028016 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e497aeb-6bbc-4c3c-908f-602edcfc1818 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:14.028286 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e497aeb-6bbc-4c3c-908f-602edcfc1818 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:14.029107 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e497aeb-6bbc-4c3c-908f-602edcfc1818 None admin] RBAC: Adding query filter params (role.id=785cca921bad4984a86e45ce6c5601c2, scope.project.id=13fba33c46cf40bbb15a08c690b4c98c, user.id=2efc792a2655439f8b8070010059b6bb) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:14.029420 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e497aeb-6bbc-4c3c-908f-602edcfc1818 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:14.030856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e497aeb-6bbc-4c3c-908f-602edcfc1818 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:14.036268 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 287/568] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:18:13 2022] GET /identity/v3/role_assignments?user.id=2efc792a2655439f8b8070010059b6bb&scope.project.id=13fba33c46cf40bbb15a08c690b4c98c&role.id=785cca921bad4984a86e45ce6c5601c2 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:14.037055 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e497aeb-6bbc-4c3c-908f-602edcfc1818 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:14.050862 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863fc3cf-860d-4130-955b-913cc5853bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:14.051682 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fc3cf-860d-4130-955b-913cc5853bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:14.052072 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fc3cf-860d-4130-955b-913cc5853bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:14.052583 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fc3cf-860d-4130-955b-913cc5853bd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:14.065507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863fc3cf-860d-4130-955b-913cc5853bd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:14.067163 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863fc3cf-860d-4130-955b-913cc5853bd6 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:14.082864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-863fc3cf-860d-4130-955b-913cc5853bd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:14.082864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:14.082864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:14.082864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:14.082864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:14.082864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:14.082864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:14.082864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:14.082864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:14.082864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:14.082864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:14.082864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:14.082864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:14.082864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:14.082864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:14.084041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:14.084041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:14.084041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:14.084041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:14.084041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:14.084041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:14.084041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:14.084041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:14.084041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:14.084041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:14.084041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:14.084041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:14.084041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:14.084041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:14.084041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:14.084041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:14.084041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:14.085164 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:14.085164 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:14.085164 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:14.085164 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:14.085771 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 282/569] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:14.086714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863fc3cf-860d-4130-955b-913cc5853bd6 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:14.100480 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:14.114978 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:14.159216 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e497aeb-6bbc-4c3c-908f-602edcfc1818 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:14.201922 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863fc3cf-860d-4130-955b-913cc5853bd6 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:14.203416 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ecbfa7-5f5d-463c-84a6-f893def514f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:14.204060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ecbfa7-5f5d-463c-84a6-f893def514f8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:14.204380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ecbfa7-5f5d-463c-84a6-f893def514f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:14.204657 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ecbfa7-5f5d-463c-84a6-f893def514f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:14.220329 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ecbfa7-5f5d-463c-84a6-f893def514f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:14.225245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-21ecbfa7-5f5d-463c-84a6-f893def514f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:14.225245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:14.225245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:14.225245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:14.225245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:14.225245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:14.225245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:14.225245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:14.225245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:14.225245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:14.225245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:14.225245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:14.225245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:14.225245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:14.225245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:14.225245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:14.226199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:14.226199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:14.226199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:14.226199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:14.226199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:14.226199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:14.226199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:14.226199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:14.226199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:14.226199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:14.226199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:14.226199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:14.226199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:14.226199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:14.226199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:14.226199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:14.227056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:14.227056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:14.227056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:14.227056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:14.227056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:14.227056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:14.227056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:14.227056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:14.227056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:14.227056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:14.227056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:14.227056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:14.227056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 288/570] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Aug 6 12:18:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:14.227857 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ecbfa7-5f5d-463c-84a6-f893def514f8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:14.244083 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63610bf5-8739-4dc7-987b-ce8450ada5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:14.244797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63610bf5-8739-4dc7-987b-ce8450ada5de None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:14.245077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63610bf5-8739-4dc7-987b-ce8450ada5de None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:14.245348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63610bf5-8739-4dc7-987b-ce8450ada5de None admin] PATH_INFO: `/v3/roles/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:14.256606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63610bf5-8739-4dc7-987b-ce8450ada5de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:14.262547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-63610bf5-8739-4dc7-987b-ce8450ada5de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:14.262547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:14.262547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:14.262547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:14.262547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:14.262547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:14.262547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:14.262547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:14.262547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:14.262547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:14.262547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:14.262547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:14.262547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:14.262547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:14.262547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:14.262547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:14.263491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:14.263491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:14.263491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:14.263491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:14.263491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:14.263491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:14.263491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:14.263491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:14.263491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:14.263491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:14.263491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:14.263491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:14.263491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:14.263491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:14.263491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:14.263491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:14.264343 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:14.264343 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:14.264343 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:14.264343 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:14.264343 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:14.264343 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:14.264343 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:14.264343 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:14.264343 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:14.264343 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:14.264343 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:14.264343 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:14.264343 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 283/571] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Aug 6 12:18:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:14.265041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63610bf5-8739-4dc7-987b-ce8450ada5de None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:14.332606 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ecbfa7-5f5d-463c-84a6-f893def514f8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:14.340070 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:14.357577 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:14.370584 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34167d95-0331-403d-a4a5-7efb35982501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:14.371200 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34167d95-0331-403d-a4a5-7efb35982501 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:14.371477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34167d95-0331-403d-a4a5-7efb35982501 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:14.371750 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34167d95-0331-403d-a4a5-7efb35982501 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:14.372467 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34167d95-0331-403d-a4a5-7efb35982501 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:14.372770 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34167d95-0331-403d-a4a5-7efb35982501 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:14.373271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63610bf5-8739-4dc7-987b-ce8450ada5de None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:14.374117 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34167d95-0331-403d-a4a5-7efb35982501 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:14.391575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 289/572] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:14.391575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34167d95-0331-403d-a4a5-7efb35982501 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:14.412507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9d3d8-c844-4659-bf72-1e9bf404910e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:14.413121 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9d3d8-c844-4659-bf72-1e9bf404910e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:14.413402 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9d3d8-c844-4659-bf72-1e9bf404910e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:14.413676 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9d3d8-c844-4659-bf72-1e9bf404910e None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:14.414364 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9d3d8-c844-4659-bf72-1e9bf404910e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:14.414665 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9d3d8-c844-4659-bf72-1e9bf404910e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:14.416076 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9d3d8-c844-4659-bf72-1e9bf404910e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:14.428346 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 284/573] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:14.428828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9d3d8-c844-4659-bf72-1e9bf404910e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:14.502219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34167d95-0331-403d-a4a5-7efb35982501 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:14.530871 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9d3d8-c844-4659-bf72-1e9bf404910e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:14.537987 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fad1ad-4162-415c-b0fa-3e6fe2835eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:14.538609 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fad1ad-4162-415c-b0fa-3e6fe2835eba None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:14.538887 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fad1ad-4162-415c-b0fa-3e6fe2835eba None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:14.539153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fad1ad-4162-415c-b0fa-3e6fe2835eba None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:14.539873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fad1ad-4162-415c-b0fa-3e6fe2835eba None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:14.540699 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fad1ad-4162-415c-b0fa-3e6fe2835eba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:14.542047 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fad1ad-4162-415c-b0fa-3e6fe2835eba None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:14.550661 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 290/574] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Aug 6 12:18:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:14.551291 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fad1ad-4162-415c-b0fa-3e6fe2835eba None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:14.566987 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:14.574019 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f72cfc-fe09-4e72-8ee9-bb8a5ad6e47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:14.574625 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f72cfc-fe09-4e72-8ee9-bb8a5ad6e47e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:14.574898 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f72cfc-fe09-4e72-8ee9-bb8a5ad6e47e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:14.575224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f72cfc-fe09-4e72-8ee9-bb8a5ad6e47e None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:14.575930 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f72cfc-fe09-4e72-8ee9-bb8a5ad6e47e None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:14.576332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f72cfc-fe09-4e72-8ee9-bb8a5ad6e47e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:14.577744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f72cfc-fe09-4e72-8ee9-bb8a5ad6e47e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:14.586514 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 285/575] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Aug 6 12:18:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:14.587582 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f72cfc-fe09-4e72-8ee9-bb8a5ad6e47e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:14.606019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:14.663290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fad1ad-4162-415c-b0fa-3e6fe2835eba None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:14.693077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f72cfc-fe09-4e72-8ee9-bb8a5ad6e47e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:14.696143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1199cfb-5416-439d-900e-07da939d9830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:14.696889 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1199cfb-5416-439d-900e-07da939d9830 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:14.697165 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1199cfb-5416-439d-900e-07da939d9830 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:14.697435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1199cfb-5416-439d-900e-07da939d9830 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:14.705175 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1199cfb-5416-439d-900e-07da939d9830 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:14.706609 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1199cfb-5416-439d-900e-07da939d9830 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:14.718477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-e1199cfb-5416-439d-900e-07da939d9830 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 06 12:18:14.718477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:14.718477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:14.718477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:14.718477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:14.718477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:14.718477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:14.718477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:14.718477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:14.718477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:14.718477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:14.718477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:14.718477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:14.718477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:14.718477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:14.718477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:14.719451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:14.719451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:14.719451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:14.719451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:14.719451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:14.719451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:14.719451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:14.719451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:14.719451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:14.719451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:14.719451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:14.719451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:14.719451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:14.719451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:14.719451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:14.719451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:14.719451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:14.719451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:14.720426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:14.720426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:14.720426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:14.720426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:14.720426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:14.720426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:14.720426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:14.720426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:14.720426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 06 12:18:14.720426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:14.720426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 291/576] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:18:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:14.720426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1199cfb-5416-439d-900e-07da939d9830 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:14.732802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c288a6-cf6f-4728-aa5d-f540df32fd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:14.733461 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c288a6-cf6f-4728-aa5d-f540df32fd55 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:14.733744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c288a6-cf6f-4728-aa5d-f540df32fd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:14.734017 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c288a6-cf6f-4728-aa5d-f540df32fd55 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:14.742059 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c288a6-cf6f-4728-aa5d-f540df32fd55 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:14.743997 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c288a6-cf6f-4728-aa5d-f540df32fd55 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:14.755348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-08c288a6-cf6f-4728-aa5d-f540df32fd55 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 12:18:14.755348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:14.755348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:14.755348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:14.755348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:14.755348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:14.755348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:14.755348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:14.755348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:14.755348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:14.755348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:14.755348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:14.755348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:14.755348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:14.755348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:14.755348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:14.756490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:14.756490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:14.756490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:14.756490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:14.756490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:14.756490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:14.756490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:14.756490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:14.756490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:14.756490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:14.756490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:14.756490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:14.756490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:14.756490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:14.756490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:14.756490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:14.756490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:14.756490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:14.757422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:14.757422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:14.757422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:14.757422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:14.757422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:14.757422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:14.757422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:14.757422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:14.757422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 12:18:14.757422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:14.757422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 286/577] 10.222.0.20 () {56 vars in 1093 bytes} [Sat Aug 6 12:18:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:14.757422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c288a6-cf6f-4728-aa5d-f540df32fd55 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:14.820540 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:14.827661 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1199cfb-5416-439d-900e-07da939d9830 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:14.835690 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:14.866499 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf513df6-2c1b-4eaf-8799-ed04f1932f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:14.867340 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf513df6-2c1b-4eaf-8799-ed04f1932f7a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:14.867780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf513df6-2c1b-4eaf-8799-ed04f1932f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:14.868208 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf513df6-2c1b-4eaf-8799-ed04f1932f7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:14.882172 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf513df6-2c1b-4eaf-8799-ed04f1932f7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:14.882801 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c288a6-cf6f-4728-aa5d-f540df32fd55 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:14.883998 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf513df6-2c1b-4eaf-8799-ed04f1932f7a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:14.899276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-cf513df6-2c1b-4eaf-8799-ed04f1932f7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:14.899276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:14.899276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:14.899276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:14.899276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:14.899276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:14.899276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:14.899276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:14.899276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:14.899276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:14.899276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:14.899276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:14.899276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:14.899276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:14.899276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:14.900903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:14.900903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:14.900903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:14.900903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:14.900903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:14.900903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:14.900903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:14.900903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:14.900903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:14.900903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:14.900903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:14.900903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:14.900903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:14.900903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:14.900903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:14.900903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:14.900903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:14.901988 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:14.901988 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:14.901988 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:14.901988 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:14.901988 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 292/578] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:14.906227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf513df6-2c1b-4eaf-8799-ed04f1932f7a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:14.921766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c4967-c33d-4da7-b0b9-94ab78bdb4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:14.922762 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c4967-c33d-4da7-b0b9-94ab78bdb4af None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:14.923045 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c4967-c33d-4da7-b0b9-94ab78bdb4af None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:14.923580 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c4967-c33d-4da7-b0b9-94ab78bdb4af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:14.936245 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c4967-c33d-4da7-b0b9-94ab78bdb4af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:14.937874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c4967-c33d-4da7-b0b9-94ab78bdb4af None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:14.951809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-7a9c4967-c33d-4da7-b0b9-94ab78bdb4af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:14.951809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:14.951809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:14.951809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:14.951809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:14.951809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:14.951809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:14.951809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:14.951809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:14.951809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:14.951809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:14.951809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:14.951809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:14.951809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:14.951809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:14.952852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:14.952852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:14.952852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:14.952852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:14.952852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:14.952852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:14.952852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:14.952852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:14.952852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:14.952852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:14.952852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:14.952852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:14.952852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:14.952852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:14.952852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:14.952852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:14.952852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:14.953749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:14.953749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:14.953749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:14.953749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:14.953749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 287/579] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:14.954098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c4967-c33d-4da7-b0b9-94ab78bdb4af None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:15.021099 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf513df6-2c1b-4eaf-8799-ed04f1932f7a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:15.058760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833bd3ac-582a-421c-b7ec-e3d2b990e7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:15.059525 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833bd3ac-582a-421c-b7ec-e3d2b990e7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:15.059725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c4967-c33d-4da7-b0b9-94ab78bdb4af None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:15.059995 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833bd3ac-582a-421c-b7ec-e3d2b990e7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:15.060406 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833bd3ac-582a-421c-b7ec-e3d2b990e7c2 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:15.061298 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833bd3ac-582a-421c-b7ec-e3d2b990e7c2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:15.061716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833bd3ac-582a-421c-b7ec-e3d2b990e7c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:15.062856 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:15.063339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833bd3ac-582a-421c-b7ec-e3d2b990e7c2 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:15.086697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 293/580] 10.222.0.20 () {56 vars in 1137 bytes} [Sat Aug 6 12:18:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:15.086697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833bd3ac-582a-421c-b7ec-e3d2b990e7c2 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:15.100673 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f7145-0b13-4be8-8039-2d7f509e291d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:15.101392 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f7145-0b13-4be8-8039-2d7f509e291d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:15.101737 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f7145-0b13-4be8-8039-2d7f509e291d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:15.102071 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f7145-0b13-4be8-8039-2d7f509e291d None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:15.103195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:15.103523 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f7145-0b13-4be8-8039-2d7f509e291d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:15.103890 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f7145-0b13-4be8-8039-2d7f509e291d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:15.105461 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f7145-0b13-4be8-8039-2d7f509e291d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:15.125484 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 288/581] 10.222.0.20 () {56 vars in 1133 bytes} [Sat Aug 6 12:18:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:15.126126 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f7145-0b13-4be8-8039-2d7f509e291d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:15.194494 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833bd3ac-582a-421c-b7ec-e3d2b990e7c2 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:15.228492 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f4eb0-ebb5-43bd-bed1-aa33555f352a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:15.229135 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f4eb0-ebb5-43bd-bed1-aa33555f352a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:15.229413 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f4eb0-ebb5-43bd-bed1-aa33555f352a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:15.229691 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f4eb0-ebb5-43bd-bed1-aa33555f352a None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:15.230372 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f4eb0-ebb5-43bd-bed1-aa33555f352a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:15.230672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f4eb0-ebb5-43bd-bed1-aa33555f352a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:15.232021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f4eb0-ebb5-43bd-bed1-aa33555f352a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:15.233061 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f7145-0b13-4be8-8039-2d7f509e291d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:15.243266 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 294/582] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:15.244060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f4eb0-ebb5-43bd-bed1-aa33555f352a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:15.273780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7018ee-aade-4146-bf18-78bd38462ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:15.274414 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7018ee-aade-4146-bf18-78bd38462ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:15.274756 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7018ee-aade-4146-bf18-78bd38462ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:15.275028 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7018ee-aade-4146-bf18-78bd38462ab1 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:15.275712 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7018ee-aade-4146-bf18-78bd38462ab1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:15.276010 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7018ee-aade-4146-bf18-78bd38462ab1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:15.277456 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7018ee-aade-4146-bf18-78bd38462ab1 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:15.288800 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 289/583] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:15.289491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7018ee-aade-4146-bf18-78bd38462ab1 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:15.314581 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:15.332130 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:15.357706 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f4eb0-ebb5-43bd-bed1-aa33555f352a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:15.393368 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5ab12-dc6c-42de-82e6-624411c3baf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:15.393965 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5ab12-dc6c-42de-82e6-624411c3baf5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:15.394215 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7018ee-aade-4146-bf18-78bd38462ab1 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:15.394335 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5ab12-dc6c-42de-82e6-624411c3baf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:15.394500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5ab12-dc6c-42de-82e6-624411c3baf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:15.405289 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed5ab12-dc6c-42de-82e6-624411c3baf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:15.407465 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed5ab12-dc6c-42de-82e6-624411c3baf5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:15.424249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-fed5ab12-dc6c-42de-82e6-624411c3baf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:15.424249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:15.424249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:15.424249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:15.424249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:15.424249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:15.424249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:15.424249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:15.424249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:15.424249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:15.424249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:15.424249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:15.424249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:15.424249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:15.424249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:15.425174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:15.425174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:15.425174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:15.425174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:15.425174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:15.425174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:15.425174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:15.425174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:15.425174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:15.425174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:15.425174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:15.425174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:15.425174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:15.425174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:15.425174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:15.425174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:15.425174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:15.426047 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:15.426047 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:15.426047 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:15.426047 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:15.426047 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 295/584] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:15.426330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5ab12-dc6c-42de-82e6-624411c3baf5 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:15.436139 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67145e3b-76da-4649-adfe-78cb9839f113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:15.436805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67145e3b-76da-4649-adfe-78cb9839f113 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:15.437083 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67145e3b-76da-4649-adfe-78cb9839f113 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:15.437349 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67145e3b-76da-4649-adfe-78cb9839f113 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:15.448005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67145e3b-76da-4649-adfe-78cb9839f113 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:15.449483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67145e3b-76da-4649-adfe-78cb9839f113 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:15.464190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-67145e3b-76da-4649-adfe-78cb9839f113 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:15.464190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:15.464190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:15.464190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:15.464190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:15.464190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:15.464190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:15.464190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:15.464190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:15.464190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:15.464190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:15.464190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:15.464190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:15.464190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:15.464190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:15.465112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:15.465112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:15.465112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:15.465112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:15.465112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:15.465112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:15.465112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:15.465112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:15.465112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:15.465112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:15.465112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:15.465112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:15.465112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:15.465112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:15.465112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:15.465112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:15.465112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:15.465991 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:15.465991 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:15.465991 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:15.465991 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:15.465991 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 290/585] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:15.466262 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67145e3b-76da-4649-adfe-78cb9839f113 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:15.533409 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5ab12-dc6c-42de-82e6-624411c3baf5 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:15.548004 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:15.563755 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:15.571802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48a339-a574-4a8b-acc4-247ffc067578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:15.572472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48a339-a574-4a8b-acc4-247ffc067578 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:15.572749 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48a339-a574-4a8b-acc4-247ffc067578 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:15.573017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48a339-a574-4a8b-acc4-247ffc067578 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:15.576056 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67145e3b-76da-4649-adfe-78cb9839f113 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:15.581521 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48a339-a574-4a8b-acc4-247ffc067578 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:15.583138 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48a339-a574-4a8b-acc4-247ffc067578 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:15.596896 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-de48a339-a574-4a8b-acc4-247ffc067578 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 12:18:15.596896 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:15.596896 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:15.596896 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:15.596896 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:15.596896 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:15.596896 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:15.596896 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:15.596896 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:15.596896 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:15.596896 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:15.596896 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:15.596896 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:15.596896 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:15.596896 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:15.596896 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:15.597841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:15.597841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:15.597841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:15.597841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:15.597841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:15.597841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:15.597841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:15.597841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:15.597841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:15.597841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:15.597841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:15.597841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:15.597841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:15.597841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:15.597841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:15.597841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:15.597841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:15.597841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:15.598763 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:15.598763 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:15.598763 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:15.598763 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:15.598763 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:15.598763 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:15.598763 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:15.598763 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:15.598763 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 12:18:15.598763 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:15.598763 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 296/586] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Aug 6 12:18:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:15.599340 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48a339-a574-4a8b-acc4-247ffc067578 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:15.616870 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c072a97-da6a-4d3c-bc7a-d76706fa77af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:15.617509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c072a97-da6a-4d3c-bc7a-d76706fa77af None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:15.617788 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c072a97-da6a-4d3c-bc7a-d76706fa77af None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:15.618062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c072a97-da6a-4d3c-bc7a-d76706fa77af None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:15.626681 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c072a97-da6a-4d3c-bc7a-d76706fa77af None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:15.628120 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c072a97-da6a-4d3c-bc7a-d76706fa77af None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:15.639653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-4c072a97-da6a-4d3c-bc7a-d76706fa77af None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 12:18:15.639653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:15.639653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:15.639653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:15.639653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:15.639653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:15.639653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:15.639653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:15.639653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:15.639653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:15.639653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:15.639653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:15.639653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:15.639653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:15.639653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:15.639653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:15.640643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:15.640643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:15.640643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:15.640643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:15.640643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:15.640643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:15.640643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:15.640643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:15.640643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:15.640643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:15.640643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:15.640643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:15.640643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:15.640643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:15.640643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:15.640643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:15.640643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:15.640643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:15.641561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:15.641561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:15.641561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:15.641561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:15.641561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:15.641561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:15.641561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:15.641561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:15.641561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 12:18:15.641561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:15.644500 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 291/587] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:18:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:15.645529 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c072a97-da6a-4d3c-bc7a-d76706fa77af None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:15.713977 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48a339-a574-4a8b-acc4-247ffc067578 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:15.749005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c072a97-da6a-4d3c-bc7a-d76706fa77af None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:15.750487 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b9c3e-62f8-42fc-bad4-8ede0626e417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:15.751434 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b9c3e-62f8-42fc-bad4-8ede0626e417 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:15.751712 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b9c3e-62f8-42fc-bad4-8ede0626e417 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:15.751981 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b9c3e-62f8-42fc-bad4-8ede0626e417 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:15.752693 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b9c3e-62f8-42fc-bad4-8ede0626e417 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:15.752989 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b9c3e-62f8-42fc-bad4-8ede0626e417 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:15.754330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b9c3e-62f8-42fc-bad4-8ede0626e417 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:15.771568 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 297/588] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:15.772331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b9c3e-62f8-42fc-bad4-8ede0626e417 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:15.786239 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd07fb-4661-484f-865d-d3902dc8dcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:15.786858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd07fb-4661-484f-865d-d3902dc8dcbb None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:15.787137 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd07fb-4661-484f-865d-d3902dc8dcbb None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:15.787407 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd07fb-4661-484f-865d-d3902dc8dcbb None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:15.788097 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd07fb-4661-484f-865d-d3902dc8dcbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:15.788439 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd07fb-4661-484f-865d-d3902dc8dcbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:15.789804 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd07fb-4661-484f-865d-d3902dc8dcbb None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:15.792136 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:15.808630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 292/589] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:15.808630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd07fb-4661-484f-865d-d3902dc8dcbb None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:15.831446 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:15.885655 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b9c3e-62f8-42fc-bad4-8ede0626e417 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:15.911370 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd07fb-4661-484f-865d-d3902dc8dcbb None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:15.920179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104fc50a-eff6-416b-98d4-0b4c2fe6f28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:15.920822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fc50a-eff6-416b-98d4-0b4c2fe6f28e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:15.921102 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fc50a-eff6-416b-98d4-0b4c2fe6f28e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:15.921375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fc50a-eff6-416b-98d4-0b4c2fe6f28e None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:15.922177 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104fc50a-eff6-416b-98d4-0b4c2fe6f28e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:15.922482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104fc50a-eff6-416b-98d4-0b4c2fe6f28e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:15.923902 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104fc50a-eff6-416b-98d4-0b4c2fe6f28e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:15.941466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 298/590] 10.222.0.20 () {56 vars in 1139 bytes} [Sat Aug 6 12:18:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:15.942286 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104fc50a-eff6-416b-98d4-0b4c2fe6f28e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:15.953026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5adf9-3f97-4ab8-9cab-f262b24683eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:15.954348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5adf9-3f97-4ab8-9cab-f262b24683eb None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:15.954628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5adf9-3f97-4ab8-9cab-f262b24683eb None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:15.954898 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5adf9-3f97-4ab8-9cab-f262b24683eb None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:15.955802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5adf9-3f97-4ab8-9cab-f262b24683eb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:15.956105 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5adf9-3f97-4ab8-9cab-f262b24683eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:15.957577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5adf9-3f97-4ab8-9cab-f262b24683eb None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:15.981706 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 293/591] 10.222.0.20 () {56 vars in 1137 bytes} [Sat Aug 6 12:18:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:15.981706 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5adf9-3f97-4ab8-9cab-f262b24683eb None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:16.032634 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:16.058888 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104fc50a-eff6-416b-98d4-0b4c2fe6f28e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:16.066074 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:16.079116 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5adf9-3f97-4ab8-9cab-f262b24683eb None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:16.093392 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347b1f6-824e-4b33-8ff9-210e5e2f4151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:16.094019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347b1f6-824e-4b33-8ff9-210e5e2f4151 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:16.094296 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347b1f6-824e-4b33-8ff9-210e5e2f4151 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:16.094564 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347b1f6-824e-4b33-8ff9-210e5e2f4151 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:16.101123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347b1f6-824e-4b33-8ff9-210e5e2f4151 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:16.103074 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347b1f6-824e-4b33-8ff9-210e5e2f4151 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:16.111834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b95c58-70c8-4cd2-900d-f6c889248ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:16.112503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b95c58-70c8-4cd2-900d-f6c889248ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:16.112780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b95c58-70c8-4cd2-900d-f6c889248ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:16.113046 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b95c58-70c8-4cd2-900d-f6c889248ae4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:16.113964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-f347b1f6-824e-4b33-8ff9-210e5e2f4151 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:16.113964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:16.113964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:16.113964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:16.113964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:16.113964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:16.113964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:16.113964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:16.113964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:16.113964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:16.113964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:16.113964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:16.113964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:16.113964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:16.113964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:16.115015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:16.115015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:16.115015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:16.115015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:16.115015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:16.115015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:16.115015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:16.115015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:16.115015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:16.115015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:16.115015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:16.115015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:16.115015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:16.115015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:16.115015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:16.115015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:16.115015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:16.116402 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:16.116402 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:16.116402 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:16.116402 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:16.116402 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:16.116402 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:16.116402 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:16.116402 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:16.116402 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 299/592] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:16.116402 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347b1f6-824e-4b33-8ff9-210e5e2f4151 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:16.120923 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b95c58-70c8-4cd2-900d-f6c889248ae4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:16.122646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b95c58-70c8-4cd2-900d-f6c889248ae4 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:16.133928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-b6b95c58-70c8-4cd2-900d-f6c889248ae4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:16.133928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:16.133928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:16.133928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:16.133928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:16.133928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:16.133928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:16.133928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:16.133928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:16.133928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:16.133928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:16.133928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:16.133928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:16.133928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:16.133928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:16.134807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:16.134807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:16.134807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:16.134807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:16.134807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:16.134807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:16.134807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:16.134807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:16.134807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:16.134807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:16.134807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:16.134807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:16.134807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:16.134807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:16.134807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:16.134807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:16.134807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:16.135700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:16.135700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:16.135700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:16.135700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:16.135700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:16.135700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:16.135700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:16.135700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:16.135700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 294/593] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:16.136199 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b95c58-70c8-4cd2-900d-f6c889248ae4 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:16.218766 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347b1f6-824e-4b33-8ff9-210e5e2f4151 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:16.231676 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b95c58-70c8-4cd2-900d-f6c889248ae4 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:16.244323 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:16.252863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30d1f5-46dc-4bc3-928e-e36cc8bae668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:16.253495 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30d1f5-46dc-4bc3-928e-e36cc8bae668 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:16.253771 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30d1f5-46dc-4bc3-928e-e36cc8bae668 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:16.254069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30d1f5-46dc-4bc3-928e-e36cc8bae668 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:16.265153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e30d1f5-46dc-4bc3-928e-e36cc8bae668 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:16.266116 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f0d47-eac1-4044-89eb-78f6c38cd85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:16.266581 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e30d1f5-46dc-4bc3-928e-e36cc8bae668 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:16.266866 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f0d47-eac1-4044-89eb-78f6c38cd85c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:16.267151 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f0d47-eac1-4044-89eb-78f6c38cd85c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:16.267428 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f0d47-eac1-4044-89eb-78f6c38cd85c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:16.278354 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f0d47-eac1-4044-89eb-78f6c38cd85c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:16.279813 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f0d47-eac1-4044-89eb-78f6c38cd85c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:16.281214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-2e30d1f5-46dc-4bc3-928e-e36cc8bae668 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:16.281214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:16.281214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:16.281214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:16.281214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:16.281214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:16.281214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:16.281214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:16.281214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:16.281214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:16.281214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:16.281214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:16.281214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:16.281214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:16.281214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:16.282143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:16.282143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:16.282143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:16.282143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:16.282143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:16.282143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:16.282143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:16.282143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:16.282143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:16.282143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:16.282143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:16.282143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:16.282143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:16.282143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:16.282143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:16.282143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:16.282143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:16.283252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:16.283252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:16.283252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:16.283252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:16.283252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 300/594] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:16.283722 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696b7c9-bb3e-4447-a17f-871211005fc9 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:16.283849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696b7c9-bb3e-4447-a17f-871211005fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:16.284102 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696b7c9-bb3e-4447-a17f-871211005fc9 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:16.288349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 301/595] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:16.288349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696b7c9-bb3e-4447-a17f-871211005fc9 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:16.296437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-0d5f0d47-eac1-4044-89eb-78f6c38cd85c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:16.296437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:16.296437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:16.296437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:16.296437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:16.296437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:16.296437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:16.296437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:16.296437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:16.296437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:16.296437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:16.296437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:16.296437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:16.296437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:16.296437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:16.297503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:16.297503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:16.297503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:16.297503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:16.297503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:16.297503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:16.297503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:16.297503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:16.297503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:16.297503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:16.297503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:16.297503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:16.297503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:16.297503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:16.297503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:16.297503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:16.297503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:16.298623 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:16.298623 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:16.298623 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:16.298623 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:16.298623 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 295/596] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:16.298623 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f0d47-eac1-4044-89eb-78f6c38cd85c None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:16.308101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:16.389113 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696b7c9-bb3e-4447-a17f-871211005fc9 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:16.397577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f0d47-eac1-4044-89eb-78f6c38cd85c None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:16.423347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d669fa-f09e-4a5b-9db9-d095a264a4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:16.423955 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d669fa-f09e-4a5b-9db9-d095a264a4b1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:16.424236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d669fa-f09e-4a5b-9db9-d095a264a4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:16.424534 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d669fa-f09e-4a5b-9db9-d095a264a4b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:16.425277 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d669fa-f09e-4a5b-9db9-d095a264a4b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:16.425581 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d669fa-f09e-4a5b-9db9-d095a264a4b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:16.426991 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d669fa-f09e-4a5b-9db9-d095a264a4b1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:16.430033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a126a-a4b6-47fc-90aa-9d910912c8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:16.431118 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a126a-a4b6-47fc-90aa-9d910912c8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:16.431399 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a126a-a4b6-47fc-90aa-9d910912c8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:16.431666 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a126a-a4b6-47fc-90aa-9d910912c8c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:16.432442 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a126a-a4b6-47fc-90aa-9d910912c8c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:16.432745 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a126a-a4b6-47fc-90aa-9d910912c8c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:16.434197 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a126a-a4b6-47fc-90aa-9d910912c8c9 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:16.439077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 302/597] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:16.439858 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d669fa-f09e-4a5b-9db9-d095a264a4b1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:16.453723 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 296/598] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:16.453723 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75e1e4-638b-468e-b973-675d2ff87fe1 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:16.453723 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75e1e4-638b-468e-b973-675d2ff87fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:16.453723 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75e1e4-638b-468e-b973-675d2ff87fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:16.469569 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-7c75e1e4-638b-468e-b973-675d2ff87fe1 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:16.470675 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7c75e1e4-638b-468e-b973-675d2ff87fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:16.487106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:16.517890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:16.537561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 297/599] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:18:16.538374 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75e1e4-638b-468e-b973-675d2ff87fe1 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:16.550617 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d669fa-f09e-4a5b-9db9-d095a264a4b1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:16.584555 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fac54-45ba-450f-9cdc-cbbacb76d6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:16.585167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fac54-45ba-450f-9cdc-cbbacb76d6b2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:16.585444 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fac54-45ba-450f-9cdc-cbbacb76d6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:16.585717 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fac54-45ba-450f-9cdc-cbbacb76d6b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:16.586401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539fac54-45ba-450f-9cdc-cbbacb76d6b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:16.586697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539fac54-45ba-450f-9cdc-cbbacb76d6b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:16.588141 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539fac54-45ba-450f-9cdc-cbbacb76d6b2 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:16.599111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 303/600] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:16.599880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fac54-45ba-450f-9cdc-cbbacb76d6b2 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:16.637568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75e1e4-638b-468e-b973-675d2ff87fe1 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:16.671103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133e37e-a692-4c0e-bfc7-6d40896e0b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:16.671784 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133e37e-a692-4c0e-bfc7-6d40896e0b1f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:16.672063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133e37e-a692-4c0e-bfc7-6d40896e0b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:16.672361 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133e37e-a692-4c0e-bfc7-6d40896e0b1f None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:16.673044 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133e37e-a692-4c0e-bfc7-6d40896e0b1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:16.673342 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133e37e-a692-4c0e-bfc7-6d40896e0b1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:16.674710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133e37e-a692-4c0e-bfc7-6d40896e0b1f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:16.689891 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 298/601] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:16.689891 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133e37e-a692-4c0e-bfc7-6d40896e0b1f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:16.700627 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fac54-45ba-450f-9cdc-cbbacb76d6b2 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:16.701898 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:16.735660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953c316-98e4-48ad-a3f0-41994a9cda14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:16.736237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953c316-98e4-48ad-a3f0-41994a9cda14 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:16.736537 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953c316-98e4-48ad-a3f0-41994a9cda14 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:16.736811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953c316-98e4-48ad-a3f0-41994a9cda14 None admin] PATH_INFO: `/v3/projects/13fba33c46cf40bbb15a08c690b4c98c/users/7610d6289efb44599d0cee03b93f9938/roles/785cca921bad4984a86e45ce6c5601c2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:16.745188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:16.752573 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953c316-98e4-48ad-a3f0-41994a9cda14 None admin] RBAC: Authorizing `identity:create_grant(project_id=13fba33c46cf40bbb15a08c690b4c98c, user_id=7610d6289efb44599d0cee03b93f9938, role_id=785cca921bad4984a86e45ce6c5601c2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:16.754013 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953c316-98e4-48ad-a3f0-41994a9cda14 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:16.766770 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 304/602] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:18:16 2022] PUT /identity/v3/projects/13fba33c46cf40bbb15a08c690b4c98c/users/7610d6289efb44599d0cee03b93f9938/roles/785cca921bad4984a86e45ce6c5601c2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:18:16.767556 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953c316-98e4-48ad-a3f0-41994a9cda14 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:16.804181 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133e37e-a692-4c0e-bfc7-6d40896e0b1f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:16.842864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af777037-cc07-48dd-8040-3664ab855c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:16.843531 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af777037-cc07-48dd-8040-3664ab855c3c None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:16.843813 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af777037-cc07-48dd-8040-3664ab855c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:16.844088 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af777037-cc07-48dd-8040-3664ab855c3c None admin] PATH_INFO: `/v3/projects/13fba33c46cf40bbb15a08c690b4c98c/users/524b728bf63c4ca88e0f25e531cf5056/roles/785cca921bad4984a86e45ce6c5601c2` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:16.860398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af777037-cc07-48dd-8040-3664ab855c3c None admin] RBAC: Authorizing `identity:create_grant(project_id=13fba33c46cf40bbb15a08c690b4c98c, user_id=524b728bf63c4ca88e0f25e531cf5056, role_id=785cca921bad4984a86e45ce6c5601c2)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:16.861852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af777037-cc07-48dd-8040-3664ab855c3c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:16.888349 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 299/603] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:18:16 2022] PUT /identity/v3/projects/13fba33c46cf40bbb15a08c690b4c98c/users/524b728bf63c4ca88e0f25e531cf5056/roles/785cca921bad4984a86e45ce6c5601c2 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:18:16.888349 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af777037-cc07-48dd-8040-3664ab855c3c None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:16.894734 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953c316-98e4-48ad-a3f0-41994a9cda14 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:16.952878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7172a-94c8-426f-98ea-75a55cf82ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:16.954026 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7172a-94c8-426f-98ea-75a55cf82ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:16.954026 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7172a-94c8-426f-98ea-75a55cf82ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:16.954195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7172a-94c8-426f-98ea-75a55cf82ec7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:16.962768 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:16.970034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df7172a-94c8-426f-98ea-75a55cf82ec7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:16.975501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-1df7172a-94c8-426f-98ea-75a55cf82ec7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:16.975501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:16.975501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:16.975501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:16.975501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:16.975501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:16.975501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:16.975501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:16.975501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:16.975501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:16.975501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:16.975501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:16.975501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:16.975501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:16.975501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:16.975501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:16.976572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:16.976572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:16.976572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:16.976572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:16.976572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:16.976572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:16.976572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:16.976572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:16.976572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:16.976572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:16.976572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:16.976572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:16.976572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:16.976572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:16.976572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:16.976572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:16.977428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:16.977428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:16.977428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:16.977428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:16.977428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:16.977428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:16.977428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:16.977428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:16.977428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:16.977428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:16.977428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:16.977428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:16.977428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 305/604] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Aug 6 12:18:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:16.986003 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7172a-94c8-426f-98ea-75a55cf82ec7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:17.012858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af777037-cc07-48dd-8040-3664ab855c3c None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:17.042125 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:17.061169 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e749bd7-ac0d-42d5-ad5f-868042a4e95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:17.061814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e749bd7-ac0d-42d5-ad5f-868042a4e95c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:17.062088 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e749bd7-ac0d-42d5-ad5f-868042a4e95c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:17.062357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e749bd7-ac0d-42d5-ad5f-868042a4e95c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:17.071339 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e749bd7-ac0d-42d5-ad5f-868042a4e95c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:17.072826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e749bd7-ac0d-42d5-ad5f-868042a4e95c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:17.085395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-7e749bd7-ac0d-42d5-ad5f-868042a4e95c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:17.085395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:17.085395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:17.085395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:17.085395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:17.085395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:17.085395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:17.085395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:17.085395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:17.085395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:17.085395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:17.085395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:17.085395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:17.085395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:17.085395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:17.086384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:17.086384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:17.086384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:17.086384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:17.086384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:17.086384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:17.086384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:17.086384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:17.086384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:17.086384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:17.086384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:17.086384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:17.086384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:17.086384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:17.086384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:17.086384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:17.086384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:17.087811 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:17.087811 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:17.087811 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:17.087811 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:17.087811 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:17.087811 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:17.087811 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:17.087811 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:17.087811 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 300/605] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:17.088289 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e749bd7-ac0d-42d5-ad5f-868042a4e95c None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:17.112358 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7172a-94c8-426f-98ea-75a55cf82ec7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:17.158885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384e74d-78bb-4b4b-b8db-c66452c48251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:17.159531 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384e74d-78bb-4b4b-b8db-c66452c48251 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:17.159810 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384e74d-78bb-4b4b-b8db-c66452c48251 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:17.160081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384e74d-78bb-4b4b-b8db-c66452c48251 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:17.169471 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384e74d-78bb-4b4b-b8db-c66452c48251 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:17.170929 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384e74d-78bb-4b4b-b8db-c66452c48251 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:17.187078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-0384e74d-78bb-4b4b-b8db-c66452c48251 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:17.187078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:17.187078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:17.187078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:17.187078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:17.187078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:17.187078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:17.187078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:17.187078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:17.187078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:17.187078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:17.187078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:17.187078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:17.187078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:17.187078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:17.188025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:17.188025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:17.188025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:17.188025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:17.188025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:17.188025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:17.188025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:17.188025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:17.188025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:17.188025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:17.188025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:17.188025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:17.188025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:17.188025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:17.188025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:17.188025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:17.188025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:17.188934 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:17.188934 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:17.188934 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:17.188934 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:17.188934 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:17.188934 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:17.188934 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:17.188934 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:17.189360 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 306/606] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:17.190515 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384e74d-78bb-4b4b-b8db-c66452c48251 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:17.200638 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e749bd7-ac0d-42d5-ad5f-868042a4e95c None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:17.225125 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:17.239248 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d52747-5a80-4a2d-b1ad-ede92642daab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:17.239909 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d52747-5a80-4a2d-b1ad-ede92642daab None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:17.240295 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d52747-5a80-4a2d-b1ad-ede92642daab None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:17.240904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d52747-5a80-4a2d-b1ad-ede92642daab None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:17.242044 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d52747-5a80-4a2d-b1ad-ede92642daab None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:17.242416 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d52747-5a80-4a2d-b1ad-ede92642daab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:17.244516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d52747-5a80-4a2d-b1ad-ede92642daab None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:17.253846 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 301/607] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Aug 6 12:18:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:17.254761 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d52747-5a80-4a2d-b1ad-ede92642daab None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:17.322138 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384e74d-78bb-4b4b-b8db-c66452c48251 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:17.333883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:17.362214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d52747-5a80-4a2d-b1ad-ede92642daab None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:17.368481 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9dae6-e7d0-421a-b5ff-539cb3f06a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:17.369181 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9dae6-e7d0-421a-b5ff-539cb3f06a89 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:17.369522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9dae6-e7d0-421a-b5ff-539cb3f06a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:17.370364 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9dae6-e7d0-421a-b5ff-539cb3f06a89 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:17.371242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9dae6-e7d0-421a-b5ff-539cb3f06a89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:17.371609 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9dae6-e7d0-421a-b5ff-539cb3f06a89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:17.374006 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9dae6-e7d0-421a-b5ff-539cb3f06a89 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:17.392551 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 307/608] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:17.393469 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9dae6-e7d0-421a-b5ff-539cb3f06a89 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:17.401581 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dda28f-8372-4fe0-9e8b-4d3c7ee5670f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:17.402390 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dda28f-8372-4fe0-9e8b-4d3c7ee5670f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:17.402673 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dda28f-8372-4fe0-9e8b-4d3c7ee5670f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:17.402939 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dda28f-8372-4fe0-9e8b-4d3c7ee5670f None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:17.404017 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dda28f-8372-4fe0-9e8b-4d3c7ee5670f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:17.404344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dda28f-8372-4fe0-9e8b-4d3c7ee5670f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:17.406363 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dda28f-8372-4fe0-9e8b-4d3c7ee5670f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:17.421785 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 302/609] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:17.422665 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dda28f-8372-4fe0-9e8b-4d3c7ee5670f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:17.474276 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:17.509881 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9dae6-e7d0-421a-b5ff-539cb3f06a89 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:17.531627 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dda28f-8372-4fe0-9e8b-4d3c7ee5670f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:17.546134 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4850c9-a175-4fb5-a9ee-ebd63cf8110a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:17.546867 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4850c9-a175-4fb5-a9ee-ebd63cf8110a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:17.547153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4850c9-a175-4fb5-a9ee-ebd63cf8110a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:17.547425 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4850c9-a175-4fb5-a9ee-ebd63cf8110a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:17.558797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4850c9-a175-4fb5-a9ee-ebd63cf8110a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:17.560493 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4850c9-a175-4fb5-a9ee-ebd63cf8110a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:17.568030 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc74757e-1af5-4f15-821f-6a2c72a7d21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:17.568794 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74757e-1af5-4f15-821f-6a2c72a7d21c None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:17.569164 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74757e-1af5-4f15-821f-6a2c72a7d21c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:17.569434 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74757e-1af5-4f15-821f-6a2c72a7d21c None admin] PATH_INFO: `/v3/projects/13fba33c46cf40bbb15a08c690b4c98c/users/55563aaa662246c5aaf107145475bd76/roles/785cca921bad4984a86e45ce6c5601c2` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:17.576276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-cd4850c9-a175-4fb5-a9ee-ebd63cf8110a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:17.576276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:17.576276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:17.576276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:17.576276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:17.576276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:17.576276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:17.576276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:17.576276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:17.576276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:17.576276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:17.576276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:17.576276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:17.576276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:17.576276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:17.578279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:17.578279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:17.578279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:17.578279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:17.578279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:17.578279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:17.578279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:17.578279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:17.578279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:17.578279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:17.578279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:17.578279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:17.578279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:17.578279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:17.578279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:17.578279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:17.578279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:17.580399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:17.580399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:17.580399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:17.580399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:17.584355 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 308/610] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:17.584355 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4850c9-a175-4fb5-a9ee-ebd63cf8110a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:17.585598 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc74757e-1af5-4f15-821f-6a2c72a7d21c None admin] RBAC: Authorizing `identity:create_grant(project_id=13fba33c46cf40bbb15a08c690b4c98c, user_id=55563aaa662246c5aaf107145475bd76, role_id=785cca921bad4984a86e45ce6c5601c2)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:17.587597 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc74757e-1af5-4f15-821f-6a2c72a7d21c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:17.604675 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 303/611] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:18:17 2022] PUT /identity/v3/projects/13fba33c46cf40bbb15a08c690b4c98c/users/55563aaa662246c5aaf107145475bd76/roles/785cca921bad4984a86e45ce6c5601c2 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:18:17.605850 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc74757e-1af5-4f15-821f-6a2c72a7d21c None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:17.608354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:17.700829 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4850c9-a175-4fb5-a9ee-ebd63cf8110a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:17.716833 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:17.723139 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc74757e-1af5-4f15-821f-6a2c72a7d21c None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:17.741624 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae9de3-8365-42e6-a855-53ef288dc5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:17.742297 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae9de3-8365-42e6-a855-53ef288dc5a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:17.742577 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae9de3-8365-42e6-a855-53ef288dc5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:17.742852 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae9de3-8365-42e6-a855-53ef288dc5a4 None admin] PATH_INFO: `/v3/projects/13fba33c46cf40bbb15a08c690b4c98c/users/d7a3cbb9c6a34fe2b280873771532e4b/roles/785cca921bad4984a86e45ce6c5601c2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:17.760166 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae9de3-8365-42e6-a855-53ef288dc5a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=13fba33c46cf40bbb15a08c690b4c98c, user_id=d7a3cbb9c6a34fe2b280873771532e4b, role_id=785cca921bad4984a86e45ce6c5601c2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:17.761595 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae9de3-8365-42e6-a855-53ef288dc5a4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:17.764202 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee62c3-39af-4d8c-8b9e-d2ac8f5a5391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:17.765192 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee62c3-39af-4d8c-8b9e-d2ac8f5a5391 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:17.765796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee62c3-39af-4d8c-8b9e-d2ac8f5a5391 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:17.766261 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee62c3-39af-4d8c-8b9e-d2ac8f5a5391 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:17.767215 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee62c3-39af-4d8c-8b9e-d2ac8f5a5391 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:17.767799 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee62c3-39af-4d8c-8b9e-d2ac8f5a5391 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:17.769571 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee62c3-39af-4d8c-8b9e-d2ac8f5a5391 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:17.784850 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 304/612] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:17.792367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 309/613] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:18:17 2022] PUT /identity/v3/projects/13fba33c46cf40bbb15a08c690b4c98c/users/d7a3cbb9c6a34fe2b280873771532e4b/roles/785cca921bad4984a86e45ce6c5601c2 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:18:17.794376 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee62c3-39af-4d8c-8b9e-d2ac8f5a5391 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:17.911214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee62c3-39af-4d8c-8b9e-d2ac8f5a5391 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:17.948513 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9627a-4a41-4f52-a93d-e4f0660e17c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:17.949219 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9627a-4a41-4f52-a93d-e4f0660e17c7 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:17.949505 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9627a-4a41-4f52-a93d-e4f0660e17c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:17.949781 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9627a-4a41-4f52-a93d-e4f0660e17c7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:17.958395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9627a-4a41-4f52-a93d-e4f0660e17c7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:17.959859 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9627a-4a41-4f52-a93d-e4f0660e17c7 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:17.978103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-20e9627a-4a41-4f52-a93d-e4f0660e17c7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 12:18:17.978103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:17.978103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:17.978103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:17.978103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:17.978103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:17.978103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:17.978103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:17.978103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:17.978103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:17.978103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:17.978103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:17.978103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:17.978103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:17.978103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:17.978103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:17.979494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:17.979494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:17.979494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:17.979494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:17.979494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:17.979494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:17.979494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:17.979494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:17.979494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:17.979494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:17.979494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:17.979494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:17.979494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:17.979494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:17.979494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:17.979494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:17.979494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:17.979494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:17.980955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:17.980955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:17.980955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:17.980955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:17.980955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:17.980955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:17.980955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:17.980955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:17.980955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 12:18:17.980955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:17.980955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 305/614] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:18:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:17.984987 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae9de3-8365-42e6-a855-53ef288dc5a4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:18.073123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:18.109636 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae9de3-8365-42e6-a855-53ef288dc5a4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:18.149306 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122faf7-785a-41b5-ae6a-9df4e2aa0c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:18.150073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122faf7-785a-41b5-ae6a-9df4e2aa0c31 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:18.150351 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122faf7-785a-41b5-ae6a-9df4e2aa0c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:18.150666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122faf7-785a-41b5-ae6a-9df4e2aa0c31 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:18.151510 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8122faf7-785a-41b5-ae6a-9df4e2aa0c31 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:18.151812 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8122faf7-785a-41b5-ae6a-9df4e2aa0c31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:18.153283 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8122faf7-785a-41b5-ae6a-9df4e2aa0c31 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:18.176017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 310/615] 10.222.0.20 () {56 vars in 1135 bytes} [Sat Aug 6 12:18:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:18.181549 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9627a-4a41-4f52-a93d-e4f0660e17c7 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:18.206484 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:18.315021 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9627a-4a41-4f52-a93d-e4f0660e17c7 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:18.358551 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433445c-ce39-4adf-9a90-94358490ce5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:18.359110 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433445c-ce39-4adf-9a90-94358490ce5f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:18.359388 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433445c-ce39-4adf-9a90-94358490ce5f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:18.359656 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433445c-ce39-4adf-9a90-94358490ce5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:18.373929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1433445c-ce39-4adf-9a90-94358490ce5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:18.375396 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1433445c-ce39-4adf-9a90-94358490ce5f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:18.389805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-1433445c-ce39-4adf-9a90-94358490ce5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:18.389805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:18.389805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:18.389805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:18.389805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:18.389805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:18.389805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:18.389805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:18.389805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:18.389805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:18.389805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:18.389805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:18.389805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:18.389805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:18.389805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:18.390708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:18.390708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:18.390708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:18.390708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:18.390708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:18.390708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:18.390708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:18.390708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:18.390708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:18.390708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:18.390708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:18.390708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:18.390708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:18.390708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:18.390708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:18.390708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:18.390708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:18.391588 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:18.391588 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:18.391588 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:18.391588 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:18.391588 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 306/616] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:18.397808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122faf7-785a-41b5-ae6a-9df4e2aa0c31 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:18.514050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122faf7-785a-41b5-ae6a-9df4e2aa0c31 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:18.546070 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:18.559310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a39b4e-ed03-4f9a-ad01-b5a6feb76075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:18.559925 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a39b4e-ed03-4f9a-ad01-b5a6feb76075 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:18.560202 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a39b4e-ed03-4f9a-ad01-b5a6feb76075 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:18.560505 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a39b4e-ed03-4f9a-ad01-b5a6feb76075 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:18.561189 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a39b4e-ed03-4f9a-ad01-b5a6feb76075 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:18.561488 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a39b4e-ed03-4f9a-ad01-b5a6feb76075 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:18.562833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a39b4e-ed03-4f9a-ad01-b5a6feb76075 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:18.574651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 311/617] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:18.580625 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433445c-ce39-4adf-9a90-94358490ce5f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:18.685879 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:18.730936 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433445c-ce39-4adf-9a90-94358490ce5f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:18.781571 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a3cf8-6f81-4190-8ed4-7e90b601a57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:18.782219 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a3cf8-6f81-4190-8ed4-7e90b601a57b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:18.782501 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a3cf8-6f81-4190-8ed4-7e90b601a57b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:18.782774 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a3cf8-6f81-4190-8ed4-7e90b601a57b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:18.793392 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a3cf8-6f81-4190-8ed4-7e90b601a57b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:18.794861 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a3cf8-6f81-4190-8ed4-7e90b601a57b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:18.806646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-ff5a3cf8-6f81-4190-8ed4-7e90b601a57b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:18.806646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:18.806646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:18.806646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:18.806646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:18.806646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:18.806646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:18.806646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:18.806646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:18.806646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:18.806646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:18.806646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:18.806646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:18.806646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:18.806646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:18.807536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:18.807536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:18.807536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:18.807536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:18.807536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:18.807536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:18.807536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:18.807536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:18.807536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:18.807536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:18.807536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:18.807536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:18.807536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:18.807536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:18.807536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:18.807536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:18.807536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:18.808474 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:18.808474 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:18.808474 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:18.808474 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:18.808474 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:18.808474 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:18.808474 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:18.808474 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:18.808474 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 307/618] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:18.813846 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a39b4e-ed03-4f9a-ad01-b5a6feb76075 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:18.948093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a39b4e-ed03-4f9a-ad01-b5a6feb76075 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:18.987805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e376b-b955-4f63-83e1-b9c7a61c441a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:18.988484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e376b-b955-4f63-83e1-b9c7a61c441a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:18.988724 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e376b-b955-4f63-83e1-b9c7a61c441a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:18.988982 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e376b-b955-4f63-83e1-b9c7a61c441a None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:18.989739 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e376b-b955-4f63-83e1-b9c7a61c441a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:18.990043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e376b-b955-4f63-83e1-b9c7a61c441a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:18.991725 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e376b-b955-4f63-83e1-b9c7a61c441a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:19.013057 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 312/619] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:19.021365 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a3cf8-6f81-4190-8ed4-7e90b601a57b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:19.154487 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6668a-af7b-481a-a100-de15d0d26f30 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:19.154866 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6668a-af7b-481a-a100-de15d0d26f30 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:19.155031 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6668a-af7b-481a-a100-de15d0d26f30 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:19.156652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 313/620] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:18:19.161572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0dc9d-4198-4489-af30-d054a2cb295d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:19.161851 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0dc9d-4198-4489-af30-d054a2cb295d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:19.162121 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0dc9d-4198-4489-af30-d054a2cb295d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:19.181827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a3cf8-6f81-4190-8ed4-7e90b601a57b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:19.196511 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e2d0dc9d-4198-4489-af30-d054a2cb295d None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:19.197699 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e2d0dc9d-4198-4489-af30-d054a2cb295d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:19.210727 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:19.222718 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fecbc-b495-402d-9b0c-ed7fc770b32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:19.223912 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fecbc-b495-402d-9b0c-ed7fc770b32e None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:19.224403 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fecbc-b495-402d-9b0c-ed7fc770b32e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:19.225320 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fecbc-b495-402d-9b0c-ed7fc770b32e None admin] PATH_INFO: `/v3/projects/13fba33c46cf40bbb15a08c690b4c98c/users/2efc792a2655439f8b8070010059b6bb/roles/785cca921bad4984a86e45ce6c5601c2` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:19.242447 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844fecbc-b495-402d-9b0c-ed7fc770b32e None admin] RBAC: Authorizing `identity:create_grant(project_id=13fba33c46cf40bbb15a08c690b4c98c, user_id=2efc792a2655439f8b8070010059b6bb, role_id=785cca921bad4984a86e45ce6c5601c2)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:19.244711 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844fecbc-b495-402d-9b0c-ed7fc770b32e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:19.264106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 308/621] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:18:19 2022] PUT /identity/v3/projects/13fba33c46cf40bbb15a08c690b4c98c/users/2efc792a2655439f8b8070010059b6bb/roles/785cca921bad4984a86e45ce6c5601c2 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:18:19.282428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:19.314223 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 314/622] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:18:19.361836 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fecbc-b495-402d-9b0c-ed7fc770b32e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:19.479851 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fecbc-b495-402d-9b0c-ed7fc770b32e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:19.511181 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482b8f7-d779-4884-8182-a89e1c145770 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:19.511467 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482b8f7-d779-4884-8182-a89e1c145770 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:19.511749 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482b8f7-d779-4884-8182-a89e1c145770 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:19.517561 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 315/623] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:19.518986 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58316ec-f9c1-43b0-bb5e-7691d2b85080 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:19.519494 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58316ec-f9c1-43b0-bb5e-7691d2b85080 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:19.519976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58316ec-f9c1-43b0-bb5e-7691d2b85080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:19.524700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731df4c-0ef4-49ab-8799-2096ecf667c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:19.525355 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731df4c-0ef4-49ab-8799-2096ecf667c9 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:19.525633 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731df4c-0ef4-49ab-8799-2096ecf667c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:19.525916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731df4c-0ef4-49ab-8799-2096ecf667c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:19.543339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f58316ec-f9c1-43b0-bb5e-7691d2b85080 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:19.544639 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731df4c-0ef4-49ab-8799-2096ecf667c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:19.545277 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f58316ec-f9c1-43b0-bb5e-7691d2b85080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:19.549929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-e731df4c-0ef4-49ab-8799-2096ecf667c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:19.549929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:19.549929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:19.549929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:19.549929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:19.549929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:19.549929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:19.549929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:19.549929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:19.549929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:19.549929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:19.549929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:19.549929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:19.549929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:19.549929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:19.549929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:19.550934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:19.550934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:19.550934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:19.550934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:19.550934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:19.550934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:19.550934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:19.550934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:19.550934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:19.550934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:19.550934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:19.550934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:19.550934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:19.550934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:19.550934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:19.550934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:19.551767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:19.551767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:19.551767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:19.551767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:19.551767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:19.551767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:19.551767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:19.551767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:19.551767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:19.551767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:19.551767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:19.551767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:19.551767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 309/624] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Aug 6 12:18:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:19.557331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731df4c-0ef4-49ab-8799-2096ecf667c9 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:19.599741 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:19.629870 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 316/625] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:18:19.670732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58316ec-f9c1-43b0-bb5e-7691d2b85080 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:19.718264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731df4c-0ef4-49ab-8799-2096ecf667c9 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:19.791247 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263f1f0-a5a8-43ac-9f9c-fe95f29bc7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:19.791975 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263f1f0-a5a8-43ac-9f9c-fe95f29bc7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:19.792262 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263f1f0-a5a8-43ac-9f9c-fe95f29bc7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:19.792558 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263f1f0-a5a8-43ac-9f9c-fe95f29bc7b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:19.793278 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2263f1f0-a5a8-43ac-9f9c-fe95f29bc7b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:19.793583 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2263f1f0-a5a8-43ac-9f9c-fe95f29bc7b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:19.794970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2263f1f0-a5a8-43ac-9f9c-fe95f29bc7b2 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:19.803835 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 310/626] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Aug 6 12:18:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:19.809896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263f1f0-a5a8-43ac-9f9c-fe95f29bc7b2 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:19.906236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58316ec-f9c1-43b0-bb5e-7691d2b85080 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:19.909095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:19.914303 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263f1f0-a5a8-43ac-9f9c-fe95f29bc7b2 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:19.921390 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:19.957191 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad20351-3445-4195-b2e7-a59af83bc7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:19.957853 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad20351-3445-4195-b2e7-a59af83bc7fe None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:19.958133 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad20351-3445-4195-b2e7-a59af83bc7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:19.958404 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad20351-3445-4195-b2e7-a59af83bc7fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:19.981132 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad20351-3445-4195-b2e7-a59af83bc7fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:19.987303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-2ad20351-3445-4195-b2e7-a59af83bc7fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:19.987303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:19.987303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:19.987303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:19.987303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:19.987303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:19.987303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:19.987303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:19.987303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:19.987303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:19.987303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:19.987303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:19.987303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:19.987303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:19.987303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:19.987303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:19.988701 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:19.988701 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:19.988701 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:19.988701 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:19.988701 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:19.988701 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:19.988701 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:19.988701 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:19.988701 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:19.988701 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:19.988701 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:19.988701 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:19.988701 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:19.988701 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:19.988701 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:19.988701 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:19.989530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:19.989530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:19.989530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:19.989530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:19.989530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:19.989530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:19.989530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:19.989530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:19.989530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:19.989530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:19.989530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:19.989530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:19.989530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 317/627] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Aug 6 12:18:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:20.004237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad20351-3445-4195-b2e7-a59af83bc7fe None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:20.006509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d0838-ef10-441a-b25b-aa18f6374688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:20.007268 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d0838-ef10-441a-b25b-aa18f6374688 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:20.007669 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d0838-ef10-441a-b25b-aa18f6374688 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:20.008039 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d0838-ef10-441a-b25b-aa18f6374688 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:20.029828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d0838-ef10-441a-b25b-aa18f6374688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:20.034662 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d0838-ef10-441a-b25b-aa18f6374688 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:20.053584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-1a7d0838-ef10-441a-b25b-aa18f6374688 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:20.053584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:20.053584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:20.053584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:20.053584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:20.053584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:20.053584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:20.053584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:20.053584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:20.053584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:20.053584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:20.053584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:20.053584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:20.053584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:20.053584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:20.054700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:20.054700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:20.054700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:20.054700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:20.054700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:20.054700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:20.054700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:20.054700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:20.054700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:20.054700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:20.054700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:20.054700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:20.054700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:20.054700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:20.054700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:20.054700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:20.054700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:20.056295 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:20.056295 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:20.056295 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:20.056295 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:20.058222 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 311/628] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:20.063785 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d0838-ef10-441a-b25b-aa18f6374688 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:20.176369 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d0838-ef10-441a-b25b-aa18f6374688 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:20.179797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad20351-3445-4195-b2e7-a59af83bc7fe None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:20.212201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624bea1-f4ce-4c16-bcca-a033da728f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:20.213015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624bea1-f4ce-4c16-bcca-a033da728f54 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:20.213469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624bea1-f4ce-4c16-bcca-a033da728f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:20.213855 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624bea1-f4ce-4c16-bcca-a033da728f54 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:20.214870 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624bea1-f4ce-4c16-bcca-a033da728f54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:20.215274 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624bea1-f4ce-4c16-bcca-a033da728f54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:20.216885 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624bea1-f4ce-4c16-bcca-a033da728f54 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:20.217137 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fc2b4-fe29-44b7-aba7-45f3d1d12e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:20.217928 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fc2b4-fe29-44b7-aba7-45f3d1d12e49 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:20.218375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fc2b4-fe29-44b7-aba7-45f3d1d12e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:20.218813 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fc2b4-fe29-44b7-aba7-45f3d1d12e49 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:20.219667 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5fc2b4-fe29-44b7-aba7-45f3d1d12e49 None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:20.220197 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5fc2b4-fe29-44b7-aba7-45f3d1d12e49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:20.221799 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5fc2b4-fe29-44b7-aba7-45f3d1d12e49 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:20.234596 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 312/629] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:20.234596 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bfa511-daa4-46c6-9288-0f808d1f19df None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:20.234596 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bfa511-daa4-46c6-9288-0f808d1f19df None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:20.234596 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bfa511-daa4-46c6-9288-0f808d1f19df None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:20.234596 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 313/630] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:20.235708 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 318/631] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Aug 6 12:18:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:20.237129 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bfa511-daa4-46c6-9288-0f808d1f19df None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:20.238908 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e2985-8f3b-43ef-86d9-d1be9c653a2b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:20.239199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e2985-8f3b-43ef-86d9-d1be9c653a2b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:20.239477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e2985-8f3b-43ef-86d9-d1be9c653a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:20.262800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-789e2985-8f3b-43ef-86d9-d1be9c653a2b None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:20.264364 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-789e2985-8f3b-43ef-86d9-d1be9c653a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:20.273792 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:20.295145 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:20.348251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 319/632] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 12:18:20.349559 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e2985-8f3b-43ef-86d9-d1be9c653a2b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:20.366410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bfa511-daa4-46c6-9288-0f808d1f19df None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:20.417651 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f48d9-27d5-4535-856c-9679972c4aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:20.418297 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f48d9-27d5-4535-856c-9679972c4aff None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:20.418594 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f48d9-27d5-4535-856c-9679972c4aff None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:20.418831 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f48d9-27d5-4535-856c-9679972c4aff None admin] PATH_INFO: `/v3/users/glance` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:20.426852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f48d9-27d5-4535-856c-9679972c4aff None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:20.428296 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f48d9-27d5-4535-856c-9679972c4aff None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:20.441138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-701f48d9-27d5-4535-856c-9679972c4aff None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 06 12:18:20.441138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:20.441138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:20.441138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:20.441138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:20.441138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:20.441138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:20.441138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:20.441138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:20.441138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:20.441138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:20.441138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:20.441138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:20.441138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:20.441138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:20.441138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:20.442117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:20.442117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:20.442117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:20.442117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:20.442117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:20.442117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:20.442117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:20.442117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:20.442117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:20.442117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:20.442117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:20.442117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:20.442117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:20.442117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:20.442117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:20.442117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:20.442117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:20.442117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:20.443095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:20.443095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:20.443095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:20.443095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:20.443095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:20.443095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:20.443095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:20.443095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:20.443095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 06 12:18:20.443095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:20.447489 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 314/633] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:18:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:20.448498 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f48d9-27d5-4535-856c-9679972c4aff None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:20.465504 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e2985-8f3b-43ef-86d9-d1be9c653a2b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:20.502864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330c7fd-a3f2-4173-b1ea-395c0c26d61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:20.503862 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330c7fd-a3f2-4173-b1ea-395c0c26d61f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:20.504393 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330c7fd-a3f2-4173-b1ea-395c0c26d61f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:20.504962 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330c7fd-a3f2-4173-b1ea-395c0c26d61f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:20.516737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330c7fd-a3f2-4173-b1ea-395c0c26d61f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:20.518510 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330c7fd-a3f2-4173-b1ea-395c0c26d61f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:20.523793 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:20.536315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-9330c7fd-a3f2-4173-b1ea-395c0c26d61f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:20.536315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:20.536315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:20.536315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:20.536315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:20.536315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:20.536315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:20.536315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:20.536315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:20.536315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:20.536315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:20.536315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:20.536315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:20.536315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:20.536315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:20.537288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:20.537288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:20.537288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:20.537288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:20.537288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:20.537288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:20.537288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:20.537288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:20.537288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:20.537288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:20.537288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:20.537288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:20.537288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:20.537288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:20.537288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:20.537288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:20.537288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:20.538176 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:20.538176 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:20.538176 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:20.538176 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:20.538176 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 320/634] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:20.538449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330c7fd-a3f2-4173-b1ea-395c0c26d61f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:20.556462 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:20.562174 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f48d9-27d5-4535-856c-9679972c4aff None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:20.596289 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1985f5d-6c80-4038-a856-8b947c03ea83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:20.596947 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1985f5d-6c80-4038-a856-8b947c03ea83 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:20.597223 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1985f5d-6c80-4038-a856-8b947c03ea83 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:20.597492 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1985f5d-6c80-4038-a856-8b947c03ea83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:20.608773 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1985f5d-6c80-4038-a856-8b947c03ea83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:20.613814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-e1985f5d-6c80-4038-a856-8b947c03ea83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:20.613814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:20.613814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:20.613814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:20.613814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:20.613814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:20.613814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:20.613814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:20.613814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:20.613814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:20.613814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:20.613814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:20.613814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:20.613814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:20.613814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:20.613814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:20.614747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:20.614747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:20.614747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:20.614747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:20.614747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:20.614747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:20.614747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:20.614747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:20.614747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:20.614747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:20.614747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:20.614747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:20.614747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:20.614747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:20.614747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:20.614747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:20.615608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:20.615608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:20.615608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:20.615608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:20.615608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:20.615608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:20.615608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:20.615608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:20.615608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:20.615608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:20.615608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:20.615608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:20.615608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 315/635] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Aug 6 12:18:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:20.617247 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6a24b-683e-4a32-8724-0b364fa2e74d None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:20.617247 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6a24b-683e-4a32-8724-0b364fa2e74d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:20.617247 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6a24b-683e-4a32-8724-0b364fa2e74d None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:20.618679 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 316/636] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:20.621183 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a6a24b-683e-4a32-8724-0b364fa2e74d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:20.656425 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330c7fd-a3f2-4173-b1ea-395c0c26d61f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:20.692784 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deaeceb-47d5-4cbf-8453-85b732c44ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:20.693407 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deaeceb-47d5-4cbf-8453-85b732c44ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:20.693684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deaeceb-47d5-4cbf-8453-85b732c44ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:20.693960 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deaeceb-47d5-4cbf-8453-85b732c44ac0 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:20.694759 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deaeceb-47d5-4cbf-8453-85b732c44ac0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:20.695062 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deaeceb-47d5-4cbf-8453-85b732c44ac0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:20.696726 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deaeceb-47d5-4cbf-8453-85b732c44ac0 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:20.715399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 321/637] 10.222.0.20 () {56 vars in 1137 bytes} [Sat Aug 6 12:18:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:20.716249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deaeceb-47d5-4cbf-8453-85b732c44ac0 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:20.741643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a6a24b-683e-4a32-8724-0b364fa2e74d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:20.770706 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:20.777014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150406e-9653-4e42-9438-fb6f7c2f102f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:20.777629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150406e-9653-4e42-9438-fb6f7c2f102f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:20.777905 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150406e-9653-4e42-9438-fb6f7c2f102f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:20.778176 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150406e-9653-4e42-9438-fb6f7c2f102f None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:20.778862 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150406e-9653-4e42-9438-fb6f7c2f102f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:20.779161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150406e-9653-4e42-9438-fb6f7c2f102f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:20.780560 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150406e-9653-4e42-9438-fb6f7c2f102f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:20.791772 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:20.796938 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 317/638] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:20.796938 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c15f6-5ff8-4c5b-a84c-c6e7c4caf1c5 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:20.796938 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c15f6-5ff8-4c5b-a84c-c6e7c4caf1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:20.796938 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c15f6-5ff8-4c5b-a84c-c6e7c4caf1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:20.812010 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-892c15f6-5ff8-4c5b-a84c-c6e7c4caf1c5 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:20.813268 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-892c15f6-5ff8-4c5b-a84c-c6e7c4caf1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:20.830247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deaeceb-47d5-4cbf-8453-85b732c44ac0 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:20.868986 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84540ee5-7dec-42fc-86d9-b03d5bd18517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:20.869528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84540ee5-7dec-42fc-86d9-b03d5bd18517 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:20.869805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84540ee5-7dec-42fc-86d9-b03d5bd18517 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:20.870076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84540ee5-7dec-42fc-86d9-b03d5bd18517 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:20.870773 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84540ee5-7dec-42fc-86d9-b03d5bd18517 None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:20.871071 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84540ee5-7dec-42fc-86d9-b03d5bd18517 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:20.872672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84540ee5-7dec-42fc-86d9-b03d5bd18517 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:20.885715 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 318/639] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:18:20.885715 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c15f6-5ff8-4c5b-a84c-c6e7c4caf1c5 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:20.885955 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 322/640] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Aug 6 12:18:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:20.885955 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84540ee5-7dec-42fc-86d9-b03d5bd18517 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:20.983722 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c15f6-5ff8-4c5b-a84c-c6e7c4caf1c5 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:20.989511 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84540ee5-7dec-42fc-86d9-b03d5bd18517 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:21.011612 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:21.025409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa53e52-75e1-4095-ab64-ddbd59ca50d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:21.026033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa53e52-75e1-4095-ab64-ddbd59ca50d5 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:21.026309 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa53e52-75e1-4095-ab64-ddbd59ca50d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:21.027054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa53e52-75e1-4095-ab64-ddbd59ca50d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:21.027192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:21.032908 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47c79b-1fa8-4e47-82fa-4f38aa5c4992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:21.033545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47c79b-1fa8-4e47-82fa-4f38aa5c4992 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:21.033823 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47c79b-1fa8-4e47-82fa-4f38aa5c4992 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:21.034096 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47c79b-1fa8-4e47-82fa-4f38aa5c4992 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:21.042340 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca47c79b-1fa8-4e47-82fa-4f38aa5c4992 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:21.043761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca47c79b-1fa8-4e47-82fa-4f38aa5c4992 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:21.054903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-ca47c79b-1fa8-4e47-82fa-4f38aa5c4992 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 12:18:21.054903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:21.054903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:21.054903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:21.054903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:21.054903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:21.054903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:21.054903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:21.054903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:21.054903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:21.054903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:21.054903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:21.054903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:21.054903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:21.054903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:21.054903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:21.055919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:21.055919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:21.055919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:21.055919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:21.055919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:21.055919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:21.055919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:21.055919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:21.055919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:21.055919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:21.055919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:21.055919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:21.055919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:21.055919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:21.055919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:21.055919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:21.055919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:21.055919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:21.057331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:21.057331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:21.057331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:21.057331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:21.057331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:21.057331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:21.057331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:21.057331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:21.057331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 12:18:21.057331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:21.057331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 323/641] 10.222.0.20 () {56 vars in 1093 bytes} [Sat Aug 6 12:18:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:21.057331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47c79b-1fa8-4e47-82fa-4f38aa5c4992 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:21.100252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa53e52-75e1-4095-ab64-ddbd59ca50d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:21.101736 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa53e52-75e1-4095-ab64-ddbd59ca50d5 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:21.161618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47c79b-1fa8-4e47-82fa-4f38aa5c4992 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:21.197690 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed349f-1107-4cbf-8511-c4a814646c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:21.198351 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed349f-1107-4cbf-8511-c4a814646c1f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:21.198626 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed349f-1107-4cbf-8511-c4a814646c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:21.198891 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed349f-1107-4cbf-8511-c4a814646c1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:21.210853 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 319/642] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:18:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:18:21.210853 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa53e52-75e1-4095-ab64-ddbd59ca50d5 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:21.213857 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed349f-1107-4cbf-8511-c4a814646c1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:21.215289 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed349f-1107-4cbf-8511-c4a814646c1f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:21.230408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-12ed349f-1107-4cbf-8511-c4a814646c1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:21.230408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:21.230408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:21.230408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:21.230408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:21.230408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:21.230408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:21.230408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:21.230408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:21.230408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:21.230408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:21.230408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:21.230408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:21.230408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:21.230408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:21.231375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:21.231375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:21.231375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:21.231375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:21.231375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:21.231375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:21.231375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:21.231375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:21.231375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:21.231375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:21.231375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:21.231375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:21.231375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:21.231375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:21.231375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:21.231375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:21.231375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:21.236871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:21.236871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:21.236871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:21.236871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:21.236871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 324/643] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:21.236871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed349f-1107-4cbf-8511-c4a814646c1f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:21.240508 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:21.287439 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:21.318961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa53e52-75e1-4095-ab64-ddbd59ca50d5 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:21.346150 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed349f-1107-4cbf-8511-c4a814646c1f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:21.351987 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097386e-c8ae-49b9-b3fd-d0bff9cfec61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:21.353165 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097386e-c8ae-49b9-b3fd-d0bff9cfec61 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:21.353446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097386e-c8ae-49b9-b3fd-d0bff9cfec61 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:21.353759 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097386e-c8ae-49b9-b3fd-d0bff9cfec61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:21.364162 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097386e-c8ae-49b9-b3fd-d0bff9cfec61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:21.370707 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-2097386e-c8ae-49b9-b3fd-d0bff9cfec61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:21.370707 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:21.370707 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:21.370707 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:21.370707 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:21.370707 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:21.370707 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:21.370707 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:21.370707 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:21.370707 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:21.370707 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:21.370707 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:21.370707 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:21.370707 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:21.370707 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:21.370707 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:21.371619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:21.371619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:21.371619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:21.371619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:21.371619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:21.371619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:21.371619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:21.371619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:21.371619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:21.371619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:21.371619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:21.371619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:21.371619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:21.371619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:21.371619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:21.371619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:21.372624 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:21.372624 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:21.372624 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:21.372624 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:21.372624 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:21.372624 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:21.372624 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:21.372624 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:21.372624 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:21.372624 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:21.372624 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:21.372624 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:21.373300 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 320/644] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Aug 6 12:18:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:21.373916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097386e-c8ae-49b9-b3fd-d0bff9cfec61 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:21.388787 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933121c9-601c-461d-a457-faea237dad63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:21.389413 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933121c9-601c-461d-a457-faea237dad63 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:21.389837 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933121c9-601c-461d-a457-faea237dad63 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:21.390105 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933121c9-601c-461d-a457-faea237dad63 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:21.390911 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933121c9-601c-461d-a457-faea237dad63 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:21.391217 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933121c9-601c-461d-a457-faea237dad63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:21.392665 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933121c9-601c-461d-a457-faea237dad63 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:21.416420 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 325/645] 10.222.0.20 () {56 vars in 1133 bytes} [Sat Aug 6 12:18:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:21.416420 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933121c9-601c-461d-a457-faea237dad63 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:21.471525 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:21.494466 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097386e-c8ae-49b9-b3fd-d0bff9cfec61 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:21.521542 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933121c9-601c-461d-a457-faea237dad63 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:21.523067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:21.534753 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bcb82c-b759-4635-a3ea-11f219e8694f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:21.535385 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bcb82c-b759-4635-a3ea-11f219e8694f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:21.535662 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bcb82c-b759-4635-a3ea-11f219e8694f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:21.535934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bcb82c-b759-4635-a3ea-11f219e8694f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:21.547732 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bcb82c-b759-4635-a3ea-11f219e8694f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:21.549236 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bcb82c-b759-4635-a3ea-11f219e8694f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:21.561257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb518e3-daab-4d06-bb8d-03f9ecf47a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:21.561878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb518e3-daab-4d06-bb8d-03f9ecf47a59 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:21.562213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb518e3-daab-4d06-bb8d-03f9ecf47a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:21.562487 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb518e3-daab-4d06-bb8d-03f9ecf47a59 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:21.563230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb518e3-daab-4d06-bb8d-03f9ecf47a59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:21.563535 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb518e3-daab-4d06-bb8d-03f9ecf47a59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:21.564431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-38bcb82c-b759-4635-a3ea-11f219e8694f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:21.564431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:21.564431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:21.564431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:21.564431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:21.564431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:21.564431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:21.564431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:21.564431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:21.564431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:21.564431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:21.564431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:21.564431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:21.564431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:21.564431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:21.565263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:21.565263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:21.565263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:21.565263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:21.565263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:21.565263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:21.565263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:21.565263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:21.565263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:21.565263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:21.565263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:21.565263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:21.565263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:21.565263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:21.565263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:21.565263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:21.565263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:21.571160 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:21.571160 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:21.571160 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:21.571160 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:21.571385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb518e3-daab-4d06-bb8d-03f9ecf47a59 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:21.571467 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 321/646] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:21.571467 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bcb82c-b759-4635-a3ea-11f219e8694f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:21.587931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 326/647] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:21.587931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb518e3-daab-4d06-bb8d-03f9ecf47a59 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:21.672521 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bcb82c-b759-4635-a3ea-11f219e8694f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:21.685617 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb518e3-daab-4d06-bb8d-03f9ecf47a59 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:21.706225 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23a6d5-8f28-49ee-9e53-b45e15d825ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:21.707151 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23a6d5-8f28-49ee-9e53-b45e15d825ef None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:21.707596 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23a6d5-8f28-49ee-9e53-b45e15d825ef None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:21.708015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23a6d5-8f28-49ee-9e53-b45e15d825ef None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:21.709095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23a6d5-8f28-49ee-9e53-b45e15d825ef None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:21.709570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23a6d5-8f28-49ee-9e53-b45e15d825ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:21.712163 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23a6d5-8f28-49ee-9e53-b45e15d825ef None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:21.714312 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:21.718662 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9179d93-4cdb-4b9f-b224-4afed683e769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:21.719277 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9179d93-4cdb-4b9f-b224-4afed683e769 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:21.719556 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9179d93-4cdb-4b9f-b224-4afed683e769 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:21.719828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9179d93-4cdb-4b9f-b224-4afed683e769 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:21.722280 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 322/648] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Aug 6 12:18:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:21.723092 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23a6d5-8f28-49ee-9e53-b45e15d825ef None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:21.754862 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:21.794404 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9179d93-4cdb-4b9f-b224-4afed683e769 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:21.795869 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9179d93-4cdb-4b9f-b224-4afed683e769 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:21.820989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23a6d5-8f28-49ee-9e53-b45e15d825ef None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:21.853601 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93679dd2-8cc4-4097-a9a2-d90244054994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:21.854210 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93679dd2-8cc4-4097-a9a2-d90244054994 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:21.854490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93679dd2-8cc4-4097-a9a2-d90244054994 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:21.854762 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93679dd2-8cc4-4097-a9a2-d90244054994 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:21.855440 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93679dd2-8cc4-4097-a9a2-d90244054994 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:21.855742 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93679dd2-8cc4-4097-a9a2-d90244054994 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:21.857129 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93679dd2-8cc4-4097-a9a2-d90244054994 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:21.868804 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 323/649] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:21.869796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93679dd2-8cc4-4097-a9a2-d90244054994 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:21.895462 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 327/650] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:18:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:18:21.896742 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9ee9d-f58b-4ca8-ab32-d9a77b76e609 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:21.897027 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9ee9d-f58b-4ca8-ab32-d9a77b76e609 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:21.897301 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9ee9d-f58b-4ca8-ab32-d9a77b76e609 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:21.898536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 328/651] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:21.899259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9ee9d-f58b-4ca8-ab32-d9a77b76e609 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:21.927569 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:21.963546 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:21.970645 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93679dd2-8cc4-4097-a9a2-d90244054994 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:22.005206 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428839e6-486d-4ae7-8f6c-39b4a0aaa5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:22.005558 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9ee9d-f58b-4ca8-ab32-d9a77b76e609 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:22.005932 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428839e6-486d-4ae7-8f6c-39b4a0aaa5db None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:22.006111 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428839e6-486d-4ae7-8f6c-39b4a0aaa5db None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:22.006386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428839e6-486d-4ae7-8f6c-39b4a0aaa5db None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:22.014552 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428839e6-486d-4ae7-8f6c-39b4a0aaa5db None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:22.016085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428839e6-486d-4ae7-8f6c-39b4a0aaa5db None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:22.027602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-428839e6-486d-4ae7-8f6c-39b4a0aaa5db None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 12:18:22.027602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:22.027602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:22.027602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:22.027602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:22.027602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:22.027602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:22.027602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:22.027602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:22.027602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:22.027602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:22.027602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:22.027602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:22.027602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:22.027602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:22.027602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:22.028574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:22.028574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:22.028574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:22.028574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:22.028574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:22.028574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:22.028574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:22.028574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:22.028574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:22.028574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:22.028574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:22.028574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:22.028574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:22.028574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:22.028574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:22.028574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:22.028574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:22.028574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:22.029496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:22.029496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:22.029496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:22.029496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:22.029496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:22.029496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:22.029496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:22.029496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:22.029496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 06 12:18:22.029496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:22.029496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 324/652] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Aug 6 12:18:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:22.030633 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428839e6-486d-4ae7-8f6c-39b4a0aaa5db None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:22.044941 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c7734-7825-48d1-9f0c-61301e451902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:22.045607 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c7734-7825-48d1-9f0c-61301e451902 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:22.045887 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c7734-7825-48d1-9f0c-61301e451902 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:22.046153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c7734-7825-48d1-9f0c-61301e451902 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:22.057452 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c7734-7825-48d1-9f0c-61301e451902 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:22.058895 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789c7734-7825-48d1-9f0c-61301e451902 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:22.073563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-789c7734-7825-48d1-9f0c-61301e451902 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:22.073563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:22.073563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:22.073563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:22.073563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:22.073563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:22.073563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:22.073563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:22.073563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:22.073563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:22.073563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:22.073563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:22.073563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:22.073563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:22.073563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:22.075256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:22.075256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:22.075256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:22.075256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:22.075256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:22.075256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:22.075256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:22.075256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:22.075256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:22.075256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:22.075256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:22.075256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:22.075256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:22.075256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:22.075256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:22.075256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:22.075256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:22.077061 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:22.077061 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:22.077061 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:22.077061 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:22.077061 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 329/653] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:22.077061 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c7734-7825-48d1-9f0c-61301e451902 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:22.139671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428839e6-486d-4ae7-8f6c-39b4a0aaa5db None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:22.155794 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:22.173962 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a9288-9b0b-450a-ba77-e80df6e5ff55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:22.174585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a9288-9b0b-450a-ba77-e80df6e5ff55 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:22.174865 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a9288-9b0b-450a-ba77-e80df6e5ff55 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:22.175135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a9288-9b0b-450a-ba77-e80df6e5ff55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:22.175470 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c7734-7825-48d1-9f0c-61301e451902 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:22.181774 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a9288-9b0b-450a-ba77-e80df6e5ff55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:22.183155 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a9288-9b0b-450a-ba77-e80df6e5ff55 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:22.188795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:22.194472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-971a9288-9b0b-450a-ba77-e80df6e5ff55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:22.194472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:22.194472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:22.194472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:22.194472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:22.194472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:22.194472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:22.194472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:22.194472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:22.194472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:22.194472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:22.194472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:22.194472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:22.194472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:22.194472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:22.195495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:22.195495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:22.195495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:22.195495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:22.195495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:22.195495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:22.195495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:22.195495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:22.195495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:22.195495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:22.195495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:22.195495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:22.195495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:22.195495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:22.195495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:22.195495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:22.195495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:22.196444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:22.196444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:22.196444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:22.196444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:22.196444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:22.196444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:22.196444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:22.196444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:22.196444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 325/654] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:22.197510 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2ed7b-428a-4355-a544-7b761d173521 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:22.197795 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2ed7b-428a-4355-a544-7b761d173521 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:22.198074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2ed7b-428a-4355-a544-7b761d173521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:22.213231 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae22a9-2f1d-403f-9cbe-297fdbd05997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:22.213864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae22a9-2f1d-403f-9cbe-297fdbd05997 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:22.214141 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae22a9-2f1d-403f-9cbe-297fdbd05997 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:22.214412 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae22a9-2f1d-403f-9cbe-297fdbd05997 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:22.215517 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-9eb2ed7b-428a-4355-a544-7b761d173521 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:22.216735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9eb2ed7b-428a-4355-a544-7b761d173521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:22.225131 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae22a9-2f1d-403f-9cbe-297fdbd05997 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:22.226557 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae22a9-2f1d-403f-9cbe-297fdbd05997 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:22.241328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-85ae22a9-2f1d-403f-9cbe-297fdbd05997 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:22.241328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:22.241328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:22.241328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:22.241328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:22.241328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:22.241328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:22.241328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:22.241328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:22.241328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:22.241328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:22.241328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:22.241328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:22.241328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:22.241328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:22.242380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:22.242380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:22.242380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:22.242380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:22.242380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:22.242380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:22.242380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:22.242380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:22.242380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:22.242380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:22.242380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:22.242380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:22.242380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:22.242380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:22.242380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:22.242380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:22.242380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:22.244011 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:22.244011 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:22.244011 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:22.244011 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:22.245482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 330/655] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:22.246254 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae22a9-2f1d-403f-9cbe-297fdbd05997 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:22.282165 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 326/656] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:18:22.282998 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2ed7b-428a-4355-a544-7b761d173521 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:22.347949 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae22a9-2f1d-403f-9cbe-297fdbd05997 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:22.384974 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b60ea-e9d3-414c-89d8-24e02874658c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:22.385581 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b60ea-e9d3-414c-89d8-24e02874658c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:22.385850 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b60ea-e9d3-414c-89d8-24e02874658c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:22.386117 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b60ea-e9d3-414c-89d8-24e02874658c None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:22.386904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b60ea-e9d3-414c-89d8-24e02874658c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:22.387202 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b60ea-e9d3-414c-89d8-24e02874658c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:22.388646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b60ea-e9d3-414c-89d8-24e02874658c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:22.400908 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2ed7b-428a-4355-a544-7b761d173521 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:22.405265 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 331/657] 10.222.0.20 () {56 vars in 1139 bytes} [Sat Aug 6 12:18:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:22.406057 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b60ea-e9d3-414c-89d8-24e02874658c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:22.411053 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:22.430135 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:22.440363 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce300b-9f58-4f1e-99e7-e3275d19f125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:22.440920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce300b-9f58-4f1e-99e7-e3275d19f125 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:22.441204 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce300b-9f58-4f1e-99e7-e3275d19f125 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:22.441478 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce300b-9f58-4f1e-99e7-e3275d19f125 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:22.442168 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce300b-9f58-4f1e-99e7-e3275d19f125 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:22.442474 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce300b-9f58-4f1e-99e7-e3275d19f125 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:22.443856 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce300b-9f58-4f1e-99e7-e3275d19f125 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:22.455641 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 327/658] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:22.456262 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce300b-9f58-4f1e-99e7-e3275d19f125 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:22.509128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b60ea-e9d3-414c-89d8-24e02874658c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:22.542454 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2520fa-8910-4691-9895-8a46c972f739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:22.543060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2520fa-8910-4691-9895-8a46c972f739 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:22.543341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2520fa-8910-4691-9895-8a46c972f739 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:22.543614 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2520fa-8910-4691-9895-8a46c972f739 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:22.544380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2520fa-8910-4691-9895-8a46c972f739 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:22.544686 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2520fa-8910-4691-9895-8a46c972f739 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:22.546091 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2520fa-8910-4691-9895-8a46c972f739 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:22.554794 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce300b-9f58-4f1e-99e7-e3275d19f125 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:22.557536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 332/659] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:22.558232 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2520fa-8910-4691-9895-8a46c972f739 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:22.588557 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90976cc6-5213-4cea-b731-f8622153fddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:22.589252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90976cc6-5213-4cea-b731-f8622153fddc None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:22.589534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90976cc6-5213-4cea-b731-f8622153fddc None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:22.589810 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90976cc6-5213-4cea-b731-f8622153fddc None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:22.590490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90976cc6-5213-4cea-b731-f8622153fddc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:22.590788 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90976cc6-5213-4cea-b731-f8622153fddc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:22.592145 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90976cc6-5213-4cea-b731-f8622153fddc None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:22.603934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 328/660] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:22.604816 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90976cc6-5213-4cea-b731-f8622153fddc None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:22.631651 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:22.642815 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:22.658817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2520fa-8910-4691-9895-8a46c972f739 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:22.695578 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2edb3-e1c4-4c60-b788-9168fcd28e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:22.696738 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2edb3-e1c4-4c60-b788-9168fcd28e03 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:22.697191 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2edb3-e1c4-4c60-b788-9168fcd28e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:22.697662 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2edb3-e1c4-4c60-b788-9168fcd28e03 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:22.705544 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90976cc6-5213-4cea-b731-f8622153fddc None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:22.710338 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad2edb3-e1c4-4c60-b788-9168fcd28e03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:22.717673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-1ad2edb3-e1c4-4c60-b788-9168fcd28e03 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:22.717673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:22.717673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:22.717673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:22.717673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:22.717673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:22.717673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:22.717673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:22.717673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:22.717673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:22.717673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:22.717673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:22.717673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:22.717673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:22.717673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:22.717673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:22.718597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:22.718597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:22.718597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:22.718597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:22.718597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:22.718597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:22.718597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:22.718597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:22.718597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:22.718597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:22.718597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:22.718597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:22.718597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:22.718597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:22.718597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:22.718597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:22.719442 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:22.719442 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:22.719442 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:22.719442 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:22.719442 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:22.719442 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:22.719442 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:22.719442 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:22.719442 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:22.719442 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:22.719442 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:22.719442 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:22.719442 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 333/661] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Aug 6 12:18:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:22.720491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2edb3-e1c4-4c60-b788-9168fcd28e03 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:22.741735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e4673-b033-4b06-a078-960e956458f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:22.742350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e4673-b033-4b06-a078-960e956458f0 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:22.742626 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e4673-b033-4b06-a078-960e956458f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:22.742929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e4673-b033-4b06-a078-960e956458f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:22.809513 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e4673-b033-4b06-a078-960e956458f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:22.810962 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e4673-b033-4b06-a078-960e956458f0 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:22.831615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2edb3-e1c4-4c60-b788-9168fcd28e03 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:22.836579 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:22.865177 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d0a25-b857-4b0a-b0e6-ee3e1bc3b363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:22.865815 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d0a25-b857-4b0a-b0e6-ee3e1bc3b363 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:22.866356 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d0a25-b857-4b0a-b0e6-ee3e1bc3b363 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:22.866633 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d0a25-b857-4b0a-b0e6-ee3e1bc3b363 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:22.874495 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d0a25-b857-4b0a-b0e6-ee3e1bc3b363 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:22.875914 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d0a25-b857-4b0a-b0e6-ee3e1bc3b363 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:22.886999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-9b0d0a25-b857-4b0a-b0e6-ee3e1bc3b363 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 12:18:22.886999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:22.886999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:22.886999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:22.886999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:22.886999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:22.886999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:22.886999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:22.886999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:22.886999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:22.886999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:22.886999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:22.886999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:22.886999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:22.886999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:22.886999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:22.888190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:22.888190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:22.888190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:22.888190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:22.888190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:22.888190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:22.888190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:22.888190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:22.888190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:22.888190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:22.888190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:22.888190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:22.888190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:22.888190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:22.888190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:22.888190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:22.888190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:22.888190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:22.889168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:22.889168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:22.889168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:22.889168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:22.889168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:22.889168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:22.889168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:22.889168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:22.889168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 12:18:22.889168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:22.889168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 334/662] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:18:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:22.889168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d0a25-b857-4b0a-b0e6-ee3e1bc3b363 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:22.892108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:22.915064 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 329/663] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:18:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:18:22.915906 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e4673-b033-4b06-a078-960e956458f0 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:22.993798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d0a25-b857-4b0a-b0e6-ee3e1bc3b363 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:23.011910 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e4673-b033-4b06-a078-960e956458f0 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:23.028961 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e93521-f88b-474f-b7cb-cb1d951ea674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:23.029613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e93521-f88b-474f-b7cb-cb1d951ea674 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:23.029843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e93521-f88b-474f-b7cb-cb1d951ea674 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:23.030117 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e93521-f88b-474f-b7cb-cb1d951ea674 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:23.030899 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e93521-f88b-474f-b7cb-cb1d951ea674 None admin] RBAC: Adding query filter params (role.id=785cca921bad4984a86e45ce6c5601c2, scope.project.id=13fba33c46cf40bbb15a08c690b4c98c, user.id=7610d6289efb44599d0cee03b93f9938) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:23.031212 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e93521-f88b-474f-b7cb-cb1d951ea674 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:23.032666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e93521-f88b-474f-b7cb-cb1d951ea674 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:23.036835 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:23.046959 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 335/664] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:18:22 2022] GET /identity/v3/role_assignments?user.id=7610d6289efb44599d0cee03b93f9938&scope.project.id=13fba33c46cf40bbb15a08c690b4c98c&role.id=785cca921bad4984a86e45ce6c5601c2 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:23.046959 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e93521-f88b-474f-b7cb-cb1d951ea674 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:23.051263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e210f88-0db1-4fbd-9f87-9b3d0db72f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:23.051927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e210f88-0db1-4fbd-9f87-9b3d0db72f4a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:23.052209 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e210f88-0db1-4fbd-9f87-9b3d0db72f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:23.052585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e210f88-0db1-4fbd-9f87-9b3d0db72f4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:23.060673 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e210f88-0db1-4fbd-9f87-9b3d0db72f4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:23.062235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e210f88-0db1-4fbd-9f87-9b3d0db72f4a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:23.075561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-8e210f88-0db1-4fbd-9f87-9b3d0db72f4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:23.075561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:23.075561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:23.075561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:23.075561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:23.075561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:23.075561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:23.075561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:23.075561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:23.075561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:23.075561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:23.075561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:23.075561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:23.075561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:23.075561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:23.077951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:23.077951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:23.077951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:23.077951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:23.077951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:23.077951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:23.077951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:23.077951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:23.077951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:23.077951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:23.077951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:23.077951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:23.077951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:23.077951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:23.077951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:23.077951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:23.077951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:23.078838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:23.078838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:23.078838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:23.078838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:23.078838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:23.078838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:23.078838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:23.078838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:23.078838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 330/665] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:23.078838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e210f88-0db1-4fbd-9f87-9b3d0db72f4a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:23.129976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:23.168889 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e93521-f88b-474f-b7cb-cb1d951ea674 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:23.189050 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e210f88-0db1-4fbd-9f87-9b3d0db72f4a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:23.207993 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1b385-59a0-4b49-8658-dd115850bfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:23.208649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1b385-59a0-4b49-8658-dd115850bfc5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:23.209728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1b385-59a0-4b49-8658-dd115850bfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:23.209728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1b385-59a0-4b49-8658-dd115850bfc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:23.209925 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b1b385-59a0-4b49-8658-dd115850bfc5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:23.210570 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b1b385-59a0-4b49-8658-dd115850bfc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:23.212727 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b1b385-59a0-4b49-8658-dd115850bfc5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:23.223913 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 336/666] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Aug 6 12:18:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:23.225008 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1b385-59a0-4b49-8658-dd115850bfc5 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:23.232272 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b7996-7bce-4f8d-9a84-490ea8ee5d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:23.232925 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b7996-7bce-4f8d-9a84-490ea8ee5d87 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:23.233201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b7996-7bce-4f8d-9a84-490ea8ee5d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:23.233471 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b7996-7bce-4f8d-9a84-490ea8ee5d87 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:23.234696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b7996-7bce-4f8d-9a84-490ea8ee5d87 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:23.234696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b7996-7bce-4f8d-9a84-490ea8ee5d87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:23.235993 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b7996-7bce-4f8d-9a84-490ea8ee5d87 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:23.257543 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 331/667] 10.222.0.20 () {56 vars in 1137 bytes} [Sat Aug 6 12:18:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:23.259168 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b7996-7bce-4f8d-9a84-490ea8ee5d87 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:23.310625 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:23.339118 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1b385-59a0-4b49-8658-dd115850bfc5 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:23.370734 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:23.374141 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b7996-7bce-4f8d-9a84-490ea8ee5d87 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:23.380847 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04209581-503e-4e74-817c-20f8d10ede32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:23.381785 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04209581-503e-4e74-817c-20f8d10ede32 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:23.382267 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04209581-503e-4e74-817c-20f8d10ede32 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:23.382544 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04209581-503e-4e74-817c-20f8d10ede32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:23.396275 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04209581-503e-4e74-817c-20f8d10ede32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:23.397759 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04209581-503e-4e74-817c-20f8d10ede32 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:23.414219 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd28e90-ff3d-4cc5-b2f4-d453b02d7b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:23.415337 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd28e90-ff3d-4cc5-b2f4-d453b02d7b9d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:23.415616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-04209581-503e-4e74-817c-20f8d10ede32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:23.415616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:23.415616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:23.415616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:23.415616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:23.415616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:23.415616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:23.415616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:23.415616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:23.415616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:23.415616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:23.415616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:23.415616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:23.415616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:23.415616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:23.416500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:23.416500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:23.416500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:23.416500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:23.416500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:23.416500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:23.416500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:23.416500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:23.416500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:23.416500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:23.416500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:23.416500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:23.416500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:23.416500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:23.416500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:23.416500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:23.416500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:23.417395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:23.417395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:23.417395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:23.417395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:23.417613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd28e90-ff3d-4cc5-b2f4-d453b02d7b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:23.417613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd28e90-ff3d-4cc5-b2f4-d453b02d7b9d None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:23.417613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd28e90-ff3d-4cc5-b2f4-d453b02d7b9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:23.417613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd28e90-ff3d-4cc5-b2f4-d453b02d7b9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:23.418425 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd28e90-ff3d-4cc5-b2f4-d453b02d7b9d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:23.421377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 337/668] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:23.422235 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04209581-503e-4e74-817c-20f8d10ede32 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:23.432224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 332/669] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:23.433136 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd28e90-ff3d-4cc5-b2f4-d453b02d7b9d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:23.530435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04209581-503e-4e74-817c-20f8d10ede32 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:23.535894 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd28e90-ff3d-4cc5-b2f4-d453b02d7b9d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:23.538484 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:23.565454 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4f5a3-ba6a-4c53-8bd1-9ef9ebf1ad81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:23.566090 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4f5a3-ba6a-4c53-8bd1-9ef9ebf1ad81 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:23.566374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4f5a3-ba6a-4c53-8bd1-9ef9ebf1ad81 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:23.566646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4f5a3-ba6a-4c53-8bd1-9ef9ebf1ad81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:23.573866 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df1ee3-c13c-4bc4-9d48-038a19cd34be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:23.574477 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df1ee3-c13c-4bc4-9d48-038a19cd34be None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:23.574750 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df1ee3-c13c-4bc4-9d48-038a19cd34be None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:23.575021 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df1ee3-c13c-4bc4-9d48-038a19cd34be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:23.578657 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf4f5a3-ba6a-4c53-8bd1-9ef9ebf1ad81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:23.580221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf4f5a3-ba6a-4c53-8bd1-9ef9ebf1ad81 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:23.594649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-6cf4f5a3-ba6a-4c53-8bd1-9ef9ebf1ad81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:23.594649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:23.594649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:23.594649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:23.594649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:23.594649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:23.594649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:23.594649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:23.594649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:23.594649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:23.594649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:23.594649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:23.594649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:23.594649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:23.594649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:23.595591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:23.595591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:23.595591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:23.595591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:23.595591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:23.595591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:23.595591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:23.595591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:23.595591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:23.595591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:23.595591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:23.595591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:23.595591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:23.595591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:23.595591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:23.595591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:23.595591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:23.601153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:23.601153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:23.601153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:23.601153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:23.601153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 338/670] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:23.601153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4f5a3-ba6a-4c53-8bd1-9ef9ebf1ad81 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:23.635150 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:23.655986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df1ee3-c13c-4bc4-9d48-038a19cd34be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:23.657479 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df1ee3-c13c-4bc4-9d48-038a19cd34be None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:23.705056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4f5a3-ba6a-4c53-8bd1-9ef9ebf1ad81 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:23.741435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af536df3-facf-434f-b158-6ec0efffd559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:23.742335 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af536df3-facf-434f-b158-6ec0efffd559 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:23.742841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af536df3-facf-434f-b158-6ec0efffd559 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:23.743791 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af536df3-facf-434f-b158-6ec0efffd559 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:23.744656 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af536df3-facf-434f-b158-6ec0efffd559 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:23.745104 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af536df3-facf-434f-b158-6ec0efffd559 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:23.746742 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af536df3-facf-434f-b158-6ec0efffd559 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:23.759092 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 339/671] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:23.759913 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af536df3-facf-434f-b158-6ec0efffd559 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:23.761358 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 333/672] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:18:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:18:23.762321 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df1ee3-c13c-4bc4-9d48-038a19cd34be None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:23.766191 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:23.869620 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:23.872072 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df1ee3-c13c-4bc4-9d48-038a19cd34be None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:23.878128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af536df3-facf-434f-b158-6ec0efffd559 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:23.908977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745df44e-13b1-455c-a2fc-bb998ce3c081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:23.909576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745df44e-13b1-455c-a2fc-bb998ce3c081 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:23.909845 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745df44e-13b1-455c-a2fc-bb998ce3c081 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:23.910119 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745df44e-13b1-455c-a2fc-bb998ce3c081 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:23.910792 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745df44e-13b1-455c-a2fc-bb998ce3c081 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:23.911090 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745df44e-13b1-455c-a2fc-bb998ce3c081 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:23.911878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b84a1d-02c3-459f-abe4-d696ae244495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:23.912479 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745df44e-13b1-455c-a2fc-bb998ce3c081 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:23.912624 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b84a1d-02c3-459f-abe4-d696ae244495 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:23.912954 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b84a1d-02c3-459f-abe4-d696ae244495 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:23.913286 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b84a1d-02c3-459f-abe4-d696ae244495 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:23.914125 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b84a1d-02c3-459f-abe4-d696ae244495 None admin] RBAC: Adding query filter params (role.id=785cca921bad4984a86e45ce6c5601c2, scope.project.id=13fba33c46cf40bbb15a08c690b4c98c, user.id=524b728bf63c4ca88e0f25e531cf5056) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:23.914486 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b84a1d-02c3-459f-abe4-d696ae244495 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:23.916023 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b84a1d-02c3-459f-abe4-d696ae244495 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:23.932562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 340/673] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:18:23 2022] GET /identity/v3/role_assignments?user.id=524b728bf63c4ca88e0f25e531cf5056&scope.project.id=13fba33c46cf40bbb15a08c690b4c98c&role.id=785cca921bad4984a86e45ce6c5601c2 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:23.932562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b84a1d-02c3-459f-abe4-d696ae244495 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:23.932844 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 334/674] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:23.932844 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745df44e-13b1-455c-a2fc-bb998ce3c081 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:24.006690 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:24.057252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b84a1d-02c3-459f-abe4-d696ae244495 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:24.060607 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745df44e-13b1-455c-a2fc-bb998ce3c081 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:24.096424 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd81ad-8321-42cb-b65f-2e2d17a4249a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:24.097234 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd81ad-8321-42cb-b65f-2e2d17a4249a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:24.097502 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd81ad-8321-42cb-b65f-2e2d17a4249a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:24.097772 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd81ad-8321-42cb-b65f-2e2d17a4249a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:24.106385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd81ad-8321-42cb-b65f-2e2d17a4249a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:24.107913 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd81ad-8321-42cb-b65f-2e2d17a4249a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:24.108798 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81ddf0-349c-4006-8a07-da4529b1d4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:24.109826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81ddf0-349c-4006-8a07-da4529b1d4df None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:24.110195 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81ddf0-349c-4006-8a07-da4529b1d4df None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:24.110838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81ddf0-349c-4006-8a07-da4529b1d4df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:24.120736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-fcdd81ad-8321-42cb-b65f-2e2d17a4249a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:24.120736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:24.120736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:24.120736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:24.120736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:24.120736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:24.120736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:24.120736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:24.120736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:24.120736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:24.120736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:24.120736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:24.120736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:24.120736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:24.120736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:24.121800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:24.121800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:24.121800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:24.121800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:24.121800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:24.121800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:24.121800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:24.121800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:24.121800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:24.121800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:24.121800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:24.121800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:24.121800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:24.121800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:24.121800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:24.121800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:24.121800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:24.122890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:24.122890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:24.122890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:24.122890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:24.122890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:24.122890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:24.122890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:24.122890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:24.122890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 341/675] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:24.123484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd81ad-8321-42cb-b65f-2e2d17a4249a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:24.126732 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81ddf0-349c-4006-8a07-da4529b1d4df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:24.128522 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81ddf0-349c-4006-8a07-da4529b1d4df None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:24.144911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-ea81ddf0-349c-4006-8a07-da4529b1d4df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:24.144911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:24.144911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:24.144911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:24.144911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:24.144911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:24.144911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:24.144911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:24.144911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:24.144911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:24.144911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:24.144911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:24.144911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:24.144911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:24.144911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:24.146027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:24.146027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:24.146027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:24.146027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:24.146027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:24.146027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:24.146027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:24.146027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:24.146027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:24.146027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:24.146027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:24.146027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:24.146027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:24.146027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:24.146027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:24.146027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:24.146027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:24.147135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:24.147135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:24.147135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:24.147135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:24.147992 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 335/676] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:24.149923 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81ddf0-349c-4006-8a07-da4529b1d4df None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:24.156142 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:24.268418 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd81ad-8321-42cb-b65f-2e2d17a4249a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:24.294598 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81ddf0-349c-4006-8a07-da4529b1d4df None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:24.304737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd392a7-5f70-4bd0-a131-c416596044dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:24.305608 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd392a7-5f70-4bd0-a131-c416596044dc None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:24.305983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd392a7-5f70-4bd0-a131-c416596044dc None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:24.306355 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd392a7-5f70-4bd0-a131-c416596044dc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:24.314366 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd392a7-5f70-4bd0-a131-c416596044dc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:24.315912 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd392a7-5f70-4bd0-a131-c416596044dc None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:24.317358 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:24.329797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-bdd392a7-5f70-4bd0-a131-c416596044dc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 12:18:24.329797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:24.329797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:24.329797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:24.329797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:24.329797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:24.329797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:24.329797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:24.329797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:24.329797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:24.329797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:24.329797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:24.329797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:24.329797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:24.329797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:24.329797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:24.331064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:24.331064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:24.331064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:24.331064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:24.331064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:24.331064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:24.331064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:24.331064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:24.331064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:24.331064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:24.331064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:24.331064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:24.331064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:24.331064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:24.331064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:24.331064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:24.331064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:24.331064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:24.332330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:24.332330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:24.332330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:24.332330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:24.332330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:24.332330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:24.332330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:24.332330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:24.332330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 12:18:24.332330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:24.334085 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 342/677] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:18:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:24.334941 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd392a7-5f70-4bd0-a131-c416596044dc None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:24.344349 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9fd27c-558b-4d3c-9850-080726c2a0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:24.345642 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fd27c-558b-4d3c-9850-080726c2a0f3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:24.346120 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fd27c-558b-4d3c-9850-080726c2a0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:24.346586 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9fd27c-558b-4d3c-9850-080726c2a0f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:24.347914 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9fd27c-558b-4d3c-9850-080726c2a0f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:24.348441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9fd27c-558b-4d3c-9850-080726c2a0f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:24.350904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9fd27c-558b-4d3c-9850-080726c2a0f3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:24.367154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 336/678] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:24.368815 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9fd27c-558b-4d3c-9850-080726c2a0f3 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:24.438567 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:24.470949 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd392a7-5f70-4bd0-a131-c416596044dc None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:24.486365 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9fd27c-558b-4d3c-9850-080726c2a0f3 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:24.519237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585d84b-a653-4a42-b4ad-3b86ded0a08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:24.520792 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585d84b-a653-4a42-b4ad-3b86ded0a08b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:24.521386 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585d84b-a653-4a42-b4ad-3b86ded0a08b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:24.521912 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585d84b-a653-4a42-b4ad-3b86ded0a08b None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:24.523035 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585d84b-a653-4a42-b4ad-3b86ded0a08b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:24.523592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585d84b-a653-4a42-b4ad-3b86ded0a08b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:24.525551 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585d84b-a653-4a42-b4ad-3b86ded0a08b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:24.525696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e7645-dd11-49da-9e8b-9f0f7c920f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:24.526386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e7645-dd11-49da-9e8b-9f0f7c920f79 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:24.526790 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e7645-dd11-49da-9e8b-9f0f7c920f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:24.527191 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e7645-dd11-49da-9e8b-9f0f7c920f79 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:24.528566 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e7645-dd11-49da-9e8b-9f0f7c920f79 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:24.529050 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e7645-dd11-49da-9e8b-9f0f7c920f79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:24.530681 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e7645-dd11-49da-9e8b-9f0f7c920f79 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:24.542756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 343/679] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:24.543800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585d84b-a653-4a42-b4ad-3b86ded0a08b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:24.554775 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 337/680] 10.222.0.20 () {56 vars in 1135 bytes} [Sat Aug 6 12:18:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:24.556288 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e7645-dd11-49da-9e8b-9f0f7c920f79 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:24.593815 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:24.667337 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585d84b-a653-4a42-b4ad-3b86ded0a08b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:24.687781 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e7645-dd11-49da-9e8b-9f0f7c920f79 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:24.704028 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a437b8-39fa-4911-95e9-6791320a1c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:24.704779 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a437b8-39fa-4911-95e9-6791320a1c89 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:24.705152 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a437b8-39fa-4911-95e9-6791320a1c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:24.705523 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a437b8-39fa-4911-95e9-6791320a1c89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:24.706453 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a437b8-39fa-4911-95e9-6791320a1c89 None admin] RBAC: Adding query filter params (role.id=785cca921bad4984a86e45ce6c5601c2, scope.project.id=13fba33c46cf40bbb15a08c690b4c98c, user.id=55563aaa662246c5aaf107145475bd76) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:24.706858 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a437b8-39fa-4911-95e9-6791320a1c89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:24.708389 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a437b8-39fa-4911-95e9-6791320a1c89 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:24.720611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 344/681] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:18:24 2022] GET /identity/v3/role_assignments?user.id=55563aaa662246c5aaf107145475bd76&scope.project.id=13fba33c46cf40bbb15a08c690b4c98c&role.id=785cca921bad4984a86e45ce6c5601c2 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:24.721705 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a437b8-39fa-4911-95e9-6791320a1c89 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:24.731042 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:24.731494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa89357-6824-4b34-a64a-ff3020928847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:24.732664 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa89357-6824-4b34-a64a-ff3020928847 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:24.734082 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa89357-6824-4b34-a64a-ff3020928847 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:24.734082 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa89357-6824-4b34-a64a-ff3020928847 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:24.743613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa89357-6824-4b34-a64a-ff3020928847 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:24.745605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa89357-6824-4b34-a64a-ff3020928847 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:24.760577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-7aa89357-6824-4b34-a64a-ff3020928847 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:24.760577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:24.760577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:24.760577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:24.760577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:24.760577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:24.760577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:24.760577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:24.760577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:24.760577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:24.760577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:24.760577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:24.760577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:24.760577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:24.760577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:24.761585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:24.761585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:24.761585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:24.761585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:24.761585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:24.761585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:24.761585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:24.761585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:24.761585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:24.761585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:24.761585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:24.761585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:24.761585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:24.761585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:24.761585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:24.761585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:24.761585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:24.762528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:24.762528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:24.762528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:24.762528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:24.762528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:24.762528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:24.762528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:24.762528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:24.765189 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 338/682] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:24.773015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa89357-6824-4b34-a64a-ff3020928847 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:24.895935 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:24.905630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa89357-6824-4b34-a64a-ff3020928847 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:24.949587 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a437b8-39fa-4911-95e9-6791320a1c89 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:24.958075 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1002d7-b30b-4002-85e5-f95fb89219a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:24.959322 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1002d7-b30b-4002-85e5-f95fb89219a5 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:24.959806 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1002d7-b30b-4002-85e5-f95fb89219a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:24.960294 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1002d7-b30b-4002-85e5-f95fb89219a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:24.961927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1002d7-b30b-4002-85e5-f95fb89219a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:24.962456 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1002d7-b30b-4002-85e5-f95fb89219a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:24.964815 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1002d7-b30b-4002-85e5-f95fb89219a5 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:24.981666 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 339/683] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:24.989344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1002d7-b30b-4002-85e5-f95fb89219a5 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:25.013542 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16cae4f-c7ba-407f-8336-056cfb8916e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:25.014272 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16cae4f-c7ba-407f-8336-056cfb8916e6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:25.014552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16cae4f-c7ba-407f-8336-056cfb8916e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:25.014819 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16cae4f-c7ba-407f-8336-056cfb8916e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:25.093752 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16cae4f-c7ba-407f-8336-056cfb8916e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:25.096051 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16cae4f-c7ba-407f-8336-056cfb8916e6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:25.100739 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:25.107301 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1002d7-b30b-4002-85e5-f95fb89219a5 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:25.151314 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306cafdc-9575-49fa-ba88-781f2affcd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:25.151938 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cafdc-9575-49fa-ba88-781f2affcd1c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:25.152219 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cafdc-9575-49fa-ba88-781f2affcd1c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:25.152519 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cafdc-9575-49fa-ba88-781f2affcd1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:25.153313 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306cafdc-9575-49fa-ba88-781f2affcd1c None admin] RBAC: Adding query filter params (role.id=785cca921bad4984a86e45ce6c5601c2, scope.project.id=13fba33c46cf40bbb15a08c690b4c98c, user.id=d7a3cbb9c6a34fe2b280873771532e4b) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:25.153712 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306cafdc-9575-49fa-ba88-781f2affcd1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:25.155197 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306cafdc-9575-49fa-ba88-781f2affcd1c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:25.161910 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 340/684] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:18:24 2022] GET /identity/v3/role_assignments?user.id=d7a3cbb9c6a34fe2b280873771532e4b&scope.project.id=13fba33c46cf40bbb15a08c690b4c98c&role.id=785cca921bad4984a86e45ce6c5601c2 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:25.179024 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f82b05-fac9-4d35-905e-34bd4c555596 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:25.179416 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f82b05-fac9-4d35-905e-34bd4c555596 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:25.179702 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f82b05-fac9-4d35-905e-34bd4c555596 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:25.181951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 341/685] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:25.187604 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f091b3-4317-4e02-b2bd-d0bc8e5b157b None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:25.188010 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f091b3-4317-4e02-b2bd-d0bc8e5b157b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:25.188294 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f091b3-4317-4e02-b2bd-d0bc8e5b157b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:25.208443 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-12f091b3-4317-4e02-b2bd-d0bc8e5b157b None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:25.209768 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-12f091b3-4317-4e02-b2bd-d0bc8e5b157b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:25.225665 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:25.242270 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 345/686] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:18:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:18:25.247274 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16cae4f-c7ba-407f-8336-056cfb8916e6 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:25.290259 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 342/687] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 12:18:25.318524 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f091b3-4317-4e02-b2bd-d0bc8e5b157b None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:25.418827 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16cae4f-c7ba-407f-8336-056cfb8916e6 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:25.420656 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:25.450026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f091b3-4317-4e02-b2bd-d0bc8e5b157b None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:25.471730 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c7426-bf21-48c3-84eb-61defa44050c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:25.472334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c7426-bf21-48c3-84eb-61defa44050c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:25.472621 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c7426-bf21-48c3-84eb-61defa44050c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:25.472894 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c7426-bf21-48c3-84eb-61defa44050c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:25.486967 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75c7426-bf21-48c3-84eb-61defa44050c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:25.488593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75c7426-bf21-48c3-84eb-61defa44050c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:25.488870 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea03647-8831-4075-be18-d1ffc5d20511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:25.489516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea03647-8831-4075-be18-d1ffc5d20511 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:25.489794 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea03647-8831-4075-be18-d1ffc5d20511 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:25.490071 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea03647-8831-4075-be18-d1ffc5d20511 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:25.505141 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea03647-8831-4075-be18-d1ffc5d20511 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:25.506597 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea03647-8831-4075-be18-d1ffc5d20511 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:25.506778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-d75c7426-bf21-48c3-84eb-61defa44050c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:25.506778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:25.506778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:25.506778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:25.506778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:25.506778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:25.506778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:25.506778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:25.506778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:25.506778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:25.506778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:25.506778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:25.506778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:25.506778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:25.506778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:25.509167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:25.509167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:25.509167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:25.509167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:25.509167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:25.509167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:25.509167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:25.509167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:25.509167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:25.509167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:25.509167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:25.509167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:25.509167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:25.509167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:25.509167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:25.509167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:25.509167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:25.510703 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:25.510703 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:25.510703 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:25.510703 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:25.510703 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 346/688] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:25.517663 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:25.518910 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c7426-bf21-48c3-84eb-61defa44050c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:25.523464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-3ea03647-8831-4075-be18-d1ffc5d20511 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:25.523464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:25.523464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:25.523464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:25.523464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:25.523464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:25.523464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:25.523464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:25.523464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:25.523464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:25.523464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:25.523464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:25.523464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:25.523464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:25.523464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:25.526113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:25.526113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:25.526113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:25.526113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:25.526113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:25.526113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:25.526113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:25.526113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:25.526113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:25.526113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:25.526113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:25.526113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:25.526113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:25.526113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:25.526113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:25.526113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:25.526113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:25.527628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:25.527628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:25.527628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:25.527628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:25.527628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 343/689] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:25.531876 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea03647-8831-4075-be18-d1ffc5d20511 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:25.624865 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c7426-bf21-48c3-84eb-61defa44050c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:25.642129 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea03647-8831-4075-be18-d1ffc5d20511 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:25.660799 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e65507-84be-44e0-b8c8-3e06be5558b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:25.661419 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e65507-84be-44e0-b8c8-3e06be5558b1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:25.661695 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e65507-84be-44e0-b8c8-3e06be5558b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:25.661966 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e65507-84be-44e0-b8c8-3e06be5558b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:25.662659 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e65507-84be-44e0-b8c8-3e06be5558b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:25.662960 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e65507-84be-44e0-b8c8-3e06be5558b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:25.665506 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e65507-84be-44e0-b8c8-3e06be5558b1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:25.682481 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 347/690] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:25.683690 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e65507-84be-44e0-b8c8-3e06be5558b1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:25.685541 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c91f8-92b6-4212-89b8-db96db62997f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:25.686162 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c91f8-92b6-4212-89b8-db96db62997f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:25.686439 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c91f8-92b6-4212-89b8-db96db62997f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:25.686709 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c91f8-92b6-4212-89b8-db96db62997f None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:25.687585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c91f8-92b6-4212-89b8-db96db62997f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:25.688751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c91f8-92b6-4212-89b8-db96db62997f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:25.689515 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c91f8-92b6-4212-89b8-db96db62997f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:25.702422 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:25.705902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 344/691] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:25.710937 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c91f8-92b6-4212-89b8-db96db62997f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:25.782810 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:25.828896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c91f8-92b6-4212-89b8-db96db62997f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:25.871748 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e65507-84be-44e0-b8c8-3e06be5558b1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:25.903272 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feda469-3916-46e5-9481-9e6173006c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:25.903894 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feda469-3916-46e5-9481-9e6173006c53 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:25.904177 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feda469-3916-46e5-9481-9e6173006c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:25.904476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feda469-3916-46e5-9481-9e6173006c53 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:25.905160 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feda469-3916-46e5-9481-9e6173006c53 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:25.906598 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feda469-3916-46e5-9481-9e6173006c53 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:25.935641 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 345/692] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Aug 6 12:18:25 2022] POST /identity/v3/users => generated 254 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 12:18:25.954875 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed20d7f-4378-456f-af39-82c1ea7c4db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:25.955577 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed20d7f-4378-456f-af39-82c1ea7c4db6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:25.955863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed20d7f-4378-456f-af39-82c1ea7c4db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:25.956138 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed20d7f-4378-456f-af39-82c1ea7c4db6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:25.969854 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed20d7f-4378-456f-af39-82c1ea7c4db6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:25.971352 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed20d7f-4378-456f-af39-82c1ea7c4db6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:25.988634 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-0ed20d7f-4378-456f-af39-82c1ea7c4db6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:25.988634 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:25.988634 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:25.988634 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:25.988634 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:25.988634 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:25.988634 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:25.988634 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:25.988634 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:25.988634 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:25.988634 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:25.988634 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:25.988634 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:25.988634 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:25.988634 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:25.989596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:25.989596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:25.989596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:25.989596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:25.989596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:25.989596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:25.989596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:25.989596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:25.989596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:25.989596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:25.989596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:25.989596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:25.989596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:25.989596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:25.989596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:25.989596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:25.989596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:25.990474 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:25.990474 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:25.990474 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:25.990474 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:25.990474 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:25.990474 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:25.990474 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:25.990474 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:25.990474 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 348/693] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:25.997250 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feda469-3916-46e5-9481-9e6173006c53 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:26.114953 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feda469-3916-46e5-9481-9e6173006c53 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:26.138644 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:26.167344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5998c932-c348-4648-b68b-654bda04e146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:26.167961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998c932-c348-4648-b68b-654bda04e146 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:26.168237 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998c932-c348-4648-b68b-654bda04e146 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:26.168537 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998c932-c348-4648-b68b-654bda04e146 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:26.169292 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5998c932-c348-4648-b68b-654bda04e146 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:26.169593 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5998c932-c348-4648-b68b-654bda04e146 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:26.171053 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5998c932-c348-4648-b68b-654bda04e146 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:26.185701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 346/694] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:26.191664 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed20d7f-4378-456f-af39-82c1ea7c4db6 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:26.277593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:26.301343 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed20d7f-4378-456f-af39-82c1ea7c4db6 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:26.341452 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f633412-b3d1-45ce-b6c2-b583337716e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:26.342080 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f633412-b3d1-45ce-b6c2-b583337716e9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:26.342361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f633412-b3d1-45ce-b6c2-b583337716e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:26.342636 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f633412-b3d1-45ce-b6c2-b583337716e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:26.343424 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f633412-b3d1-45ce-b6c2-b583337716e9 None admin] RBAC: Adding query filter params (role.id=785cca921bad4984a86e45ce6c5601c2, scope.project.id=13fba33c46cf40bbb15a08c690b4c98c, user.id=2efc792a2655439f8b8070010059b6bb) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:26.343733 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f633412-b3d1-45ce-b6c2-b583337716e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:26.345357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f633412-b3d1-45ce-b6c2-b583337716e9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:26.356426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 349/695] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:18:26 2022] GET /identity/v3/role_assignments?user.id=2efc792a2655439f8b8070010059b6bb&scope.project.id=13fba33c46cf40bbb15a08c690b4c98c&role.id=785cca921bad4984a86e45ce6c5601c2 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:26.382774 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e2287-ec5c-4a93-ae5f-100ee139e335 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:26.383283 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e2287-ec5c-4a93-ae5f-100ee139e335 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:26.383283 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e2287-ec5c-4a93-ae5f-100ee139e335 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:26.385214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 347/696] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:26.392224 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e4146-42ee-4a07-ba78-e6b42e99bf1b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:26.392572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e4146-42ee-4a07-ba78-e6b42e99bf1b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:26.392834 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e4146-42ee-4a07-ba78-e6b42e99bf1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:26.412357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c77e4146-42ee-4a07-ba78-e6b42e99bf1b None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:26.413619 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c77e4146-42ee-4a07-ba78-e6b42e99bf1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:26.495940 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 350/697] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:18:26.520006 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e2287-ec5c-4a93-ae5f-100ee139e335 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:26.628570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e2287-ec5c-4a93-ae5f-100ee139e335 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:26.779971 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b43f88-0835-4d5a-8569-83451656fd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:26.780573 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b43f88-0835-4d5a-8569-83451656fd6f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:26.780852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b43f88-0835-4d5a-8569-83451656fd6f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:26.781138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b43f88-0835-4d5a-8569-83451656fd6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:26.792371 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b43f88-0835-4d5a-8569-83451656fd6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:26.797984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-d4b43f88-0835-4d5a-8569-83451656fd6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:18:26.797984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:26.797984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:26.797984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:26.797984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:26.797984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:26.797984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:26.797984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:26.797984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:26.797984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:26.797984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:26.797984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:26.797984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:26.797984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:26.797984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:26.797984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:26.798920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:26.798920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:26.798920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:26.798920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:26.798920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:26.798920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:26.798920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:26.798920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:26.798920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:26.798920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:26.798920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:26.798920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:26.798920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:26.798920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:26.798920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:26.798920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:26.799761 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:26.799761 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:26.799761 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:26.799761 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:26.799761 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:26.799761 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:26.799761 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:26.799761 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:26.799761 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:26.799761 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:26.799761 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:18:26.799761 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:26.802006 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 348/698] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:18:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:26.808015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e4146-42ee-4a07-ba78-e6b42e99bf1b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:26.879494 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:26.914034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e4146-42ee-4a07-ba78-e6b42e99bf1b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:26.949329 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9f700-07c3-44a5-8736-71b71cb70a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:26.950021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9f700-07c3-44a5-8736-71b71cb70a06 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:26.950307 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9f700-07c3-44a5-8736-71b71cb70a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:26.950573 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9f700-07c3-44a5-8736-71b71cb70a06 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:26.951303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9f700-07c3-44a5-8736-71b71cb70a06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:26.951606 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9f700-07c3-44a5-8736-71b71cb70a06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:26.952996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f9f700-07c3-44a5-8736-71b71cb70a06 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:26.962505 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 351/699] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:18:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:26.967870 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b43f88-0835-4d5a-8569-83451656fd6f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:27.022728 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:27.117021 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b43f88-0835-4d5a-8569-83451656fd6f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:27.162423 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60d97c-4dd5-40df-9ab5-7c20d9e281e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:27.163357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60d97c-4dd5-40df-9ab5-7c20d9e281e6 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:27.163777 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60d97c-4dd5-40df-9ab5-7c20d9e281e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:27.164189 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60d97c-4dd5-40df-9ab5-7c20d9e281e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:27.181199 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d60d97c-4dd5-40df-9ab5-7c20d9e281e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:27.182859 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d60d97c-4dd5-40df-9ab5-7c20d9e281e6 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:27.194802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82c971-861a-4551-8c2d-c36dc25ca159 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:27.195292 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82c971-861a-4551-8c2d-c36dc25ca159 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:27.195376 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82c971-861a-4551-8c2d-c36dc25ca159 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:27.197499 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 352/700] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:27.202311 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82f8ae-ee87-4eba-9f14-855056421627 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:27.202592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82f8ae-ee87-4eba-9f14-855056421627 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:27.202864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82f8ae-ee87-4eba-9f14-855056421627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:27.203696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-4d60d97c-4dd5-40df-9ab5-7c20d9e281e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:27.203696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:27.203696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:27.203696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:27.203696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:27.203696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:27.203696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:27.203696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:27.203696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:27.203696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:27.203696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:27.203696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:27.203696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:27.203696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:27.203696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:27.204738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:27.204738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:27.204738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:27.204738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:27.204738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:27.204738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:27.204738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:27.204738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:27.204738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:27.204738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:27.204738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:27.204738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:27.204738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:27.204738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:27.204738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:27.204738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:27.204738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:27.205877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:27.205877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:27.205877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:27.205877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:27.211315 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 349/701] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:27.221859 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60d97c-4dd5-40df-9ab5-7c20d9e281e6 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:27.226791 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6b82f8ae-ee87-4eba-9f14-855056421627 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:27.228663 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6b82f8ae-ee87-4eba-9f14-855056421627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:27.298345 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 353/702] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 06 12:18:27.321975 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82f8ae-ee87-4eba-9f14-855056421627 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:27.365940 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:27.382404 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:27.393472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60d97c-4dd5-40df-9ab5-7c20d9e281e6 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:27.424648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82f8ae-ee87-4eba-9f14-855056421627 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:27.460400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c66ef0-4b37-4bdc-97f7-3e7f8d5e1935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:27.461196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c66ef0-4b37-4bdc-97f7-3e7f8d5e1935 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:27.462250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c66ef0-4b37-4bdc-97f7-3e7f8d5e1935 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:27.462620 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c66ef0-4b37-4bdc-97f7-3e7f8d5e1935 None admin] PATH_INFO: `/v3/services/network` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:27.463321 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5179b-9b4b-4b67-864b-27c58ea2315e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:27.463451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c66ef0-4b37-4bdc-97f7-3e7f8d5e1935 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:27.464245 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5179b-9b4b-4b67-864b-27c58ea2315e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:27.468324 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c66ef0-4b37-4bdc-97f7-3e7f8d5e1935 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:27.468622 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5179b-9b4b-4b67-864b-27c58ea2315e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:27.468916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5179b-9b4b-4b67-864b-27c58ea2315e None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:27.469849 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5179b-9b4b-4b67-864b-27c58ea2315e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:27.470144 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5179b-9b4b-4b67-864b-27c58ea2315e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:27.471607 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5179b-9b4b-4b67-864b-27c58ea2315e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:27.484840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-69c66ef0-4b37-4bdc-97f7-3e7f8d5e1935 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 12:18:27.484840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:27.484840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:18:27.484840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:18:27.484840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:18:27.484840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:18:27.484840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:18:27.484840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:27.484840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 12:18:27.484840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:27.484840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:18:27.484840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:27.484840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:27.484840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:27.484840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:27.484840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:27.484840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:27.485931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:27.485931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:27.485931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:27.485931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:27.485931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:27.485931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:27.485931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:18:27.485931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:18:27.485931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:18:27.485931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:18:27.485931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:27.485931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:27.485931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:27.485931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:27.485931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:27.485931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:27.486789 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:27.486789 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:27.486789 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:27.486789 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:27.486789 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:27.486789 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:27.486789 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:27.486789 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:27.486789 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:27.486789 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:27.486789 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:18:27.486789 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:27.486789 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 12:18:27.486789 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:27.488109 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 354/703] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:27 2022] GET /identity/v3/services/network => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:27.497975 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c66ef0-4b37-4bdc-97f7-3e7f8d5e1935 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:27.506806 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 350/704] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:27.517776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5179b-9b4b-4b67-864b-27c58ea2315e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:27.644190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c66ef0-4b37-4bdc-97f7-3e7f8d5e1935 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:27.668216 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:27.679341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d638df-f9ca-435c-a8e1-f1a767e16fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:27.679906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d638df-f9ca-435c-a8e1-f1a767e16fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:27.680193 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d638df-f9ca-435c-a8e1-f1a767e16fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:27.680987 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d638df-f9ca-435c-a8e1-f1a767e16fb6 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:27.681815 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d638df-f9ca-435c-a8e1-f1a767e16fb6 None admin] RBAC: Adding query filter params (name=network) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:27.682513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d638df-f9ca-435c-a8e1-f1a767e16fb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:27.683965 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d638df-f9ca-435c-a8e1-f1a767e16fb6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:27.689061 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 355/705] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Aug 6 12:18:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:27.694471 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d638df-f9ca-435c-a8e1-f1a767e16fb6 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:27.765600 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5179b-9b4b-4b67-864b-27c58ea2315e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:27.838567 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d23cc9-3e15-4cdc-a15f-4a97c9012fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:27.839214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d23cc9-3e15-4cdc-a15f-4a97c9012fcf None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:27.839496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d23cc9-3e15-4cdc-a15f-4a97c9012fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:27.839767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d23cc9-3e15-4cdc-a15f-4a97c9012fcf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:27.841167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d638df-f9ca-435c-a8e1-f1a767e16fb6 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:27.852017 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d23cc9-3e15-4cdc-a15f-4a97c9012fcf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:27.861738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d23cc9-3e15-4cdc-a15f-4a97c9012fcf None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:27.876953 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca92f4-a21c-40e7-b308-494bd96bd48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:27.877570 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca92f4-a21c-40e7-b308-494bd96bd48f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:27.877849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca92f4-a21c-40e7-b308-494bd96bd48f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:27.878122 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca92f4-a21c-40e7-b308-494bd96bd48f None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:27.878804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca92f4-a21c-40e7-b308-494bd96bd48f None admin] RBAC: Adding query filter params (type=network) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:27.879103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca92f4-a21c-40e7-b308-494bd96bd48f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:27.880488 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca92f4-a21c-40e7-b308-494bd96bd48f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:27.887098 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 356/706] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Aug 6 12:18:27 2022] GET /identity/v3/services?type=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:27.888398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-63d23cc9-3e15-4cdc-a15f-4a97c9012fcf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 12:18:27.888398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:27.888398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:27.888398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:27.888398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:27.888398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:27.888398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:27.888398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:27.888398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:27.888398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:27.888398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:27.888398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:27.888398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:27.888398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:27.888398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:27.888398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:27.889325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:27.889325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:27.889325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:27.889325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:27.889325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:27.889325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:27.889325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:27.889325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:27.889325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:27.889325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:27.889325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:27.889325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:27.889325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:27.889325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:27.889325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:27.889325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:27.889325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:27.889325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:27.890834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:27.890834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:27.890834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:27.890834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:27.890834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:27.890834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:27.890834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:27.890834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:27.890834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 12:18:27.890834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:27.890834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 351/707] 10.222.0.20 () {56 vars in 1093 bytes} [Sat Aug 6 12:18:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:27.895031 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca92f4-a21c-40e7-b308-494bd96bd48f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:27.982167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:28.003850 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54540f00-8f1d-41b6-8094-b2acb619f05e None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:28.004085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54540f00-8f1d-41b6-8094-b2acb619f05e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:28.004374 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54540f00-8f1d-41b6-8094-b2acb619f05e None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:28.005883 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 352/708] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:28.012011 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021966e-c411-4b78-900e-4b17433009f6 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:28.012353 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021966e-c411-4b78-900e-4b17433009f6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:28.012601 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021966e-c411-4b78-900e-4b17433009f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:28.047319 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-8021966e-c411-4b78-900e-4b17433009f6 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:28.049846 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8021966e-c411-4b78-900e-4b17433009f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:28.052411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:28.100064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca92f4-a21c-40e7-b308-494bd96bd48f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:28.148376 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a30a2-9eb1-4a09-9b46-8a682cc47466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:28.149508 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a30a2-9eb1-4a09-9b46-8a682cc47466 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:28.149910 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a30a2-9eb1-4a09-9b46-8a682cc47466 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:28.150183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a30a2-9eb1-4a09-9b46-8a682cc47466 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:28.151032 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a30a2-9eb1-4a09-9b46-8a682cc47466 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:28.151337 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a30a2-9eb1-4a09-9b46-8a682cc47466 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:28.152844 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a30a2-9eb1-4a09-9b46-8a682cc47466 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:28.159248 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 353/709] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:18:28.175080 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 357/710] 10.222.0.20 () {56 vars in 1133 bytes} [Sat Aug 6 12:18:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:28.181314 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8021966e-c411-4b78-900e-4b17433009f6 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:28.188979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a30a2-9eb1-4a09-9b46-8a682cc47466 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:28.297164 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a30a2-9eb1-4a09-9b46-8a682cc47466 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:28.308869 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:28.329288 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8021966e-c411-4b78-900e-4b17433009f6 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:28.352250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b6c63-7d9d-4e8c-8cff-dc8f24d9ff31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:28.352923 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b6c63-7d9d-4e8c-8cff-dc8f24d9ff31 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:28.353198 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b6c63-7d9d-4e8c-8cff-dc8f24d9ff31 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:28.353467 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b6c63-7d9d-4e8c-8cff-dc8f24d9ff31 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:28.354094 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b6c63-7d9d-4e8c-8cff-dc8f24d9ff31 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:28.355450 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b6c63-7d9d-4e8c-8cff-dc8f24d9ff31 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:28.375377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-2a7b6c63-7d9d-4e8c-8cff-dc8f24d9ff31 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 12:18:28.375377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:28.375377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:18:28.375377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:18:28.375377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:18:28.375377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:18:28.375377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:18:28.375377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:28.375377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 12:18:28.375377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:28.375377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:18:28.375377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:28.375377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:28.375377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:28.375377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:28.375377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:28.376385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:28.376385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:28.376385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:28.376385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:28.376385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:28.376385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:28.376385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:28.376385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:18:28.376385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:18:28.376385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:18:28.376385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:18:28.376385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:28.376385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:28.376385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:28.376385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:28.376385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:28.377239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:28.377239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:28.377239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:28.377239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:28.377239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:28.377239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:28.377239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:28.377239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:28.377239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:28.377239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:28.377239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:28.377239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:18:28.377239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:28.377239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 12:18:28.377239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:28.377239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 358/711] 10.222.0.20 () {56 vars in 1129 bytes} [Sat Aug 6 12:18:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:28.382663 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b6c63-7d9d-4e8c-8cff-dc8f24d9ff31 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:28.397697 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343257b-1fe5-4642-b05b-8335c17f2a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:28.398320 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343257b-1fe5-4642-b05b-8335c17f2a59 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:28.398628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343257b-1fe5-4642-b05b-8335c17f2a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:28.398840 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343257b-1fe5-4642-b05b-8335c17f2a59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:28.416506 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:28.516952 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343257b-1fe5-4642-b05b-8335c17f2a59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:28.518635 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343257b-1fe5-4642-b05b-8335c17f2a59 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:28.572961 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b6c63-7d9d-4e8c-8cff-dc8f24d9ff31 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:28.608714 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0686086-f357-4d22-b886-7aef062eb4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:28.609744 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0686086-f357-4d22-b886-7aef062eb4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:28.610270 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0686086-f357-4d22-b886-7aef062eb4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:28.610857 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0686086-f357-4d22-b886-7aef062eb4d2 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:28.611674 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0686086-f357-4d22-b886-7aef062eb4d2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:28.612153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0686086-f357-4d22-b886-7aef062eb4d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:28.613646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0686086-f357-4d22-b886-7aef062eb4d2 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:28.622588 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 359/712] 10.222.0.20 () {56 vars in 1124 bytes} [Sat Aug 6 12:18:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:28.622588 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3f17c-834e-4584-82e5-1f2e5f2557b9 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:28.622588 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3f17c-834e-4584-82e5-1f2e5f2557b9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:28.622588 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3f17c-834e-4584-82e5-1f2e5f2557b9 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:28.622588 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 360/713] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:28.634422 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e21715-25c8-43ef-9aea-079d3231ea00 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:28.634709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e21715-25c8-43ef-9aea-079d3231ea00 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:28.634980 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e21715-25c8-43ef-9aea-079d3231ea00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:28.646199 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 354/714] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:18:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:18:28.647272 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343257b-1fe5-4642-b05b-8335c17f2a59 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:28.657732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f6e21715-25c8-43ef-9aea-079d3231ea00 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:28.658848 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f6e21715-25c8-43ef-9aea-079d3231ea00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:28.689655 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:28.703379 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:28.734304 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 361/715] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:18:28.735189 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e21715-25c8-43ef-9aea-079d3231ea00 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:28.780072 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343257b-1fe5-4642-b05b-8335c17f2a59 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:28.823700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd884c-5e76-4c78-b8a8-510ab8105fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:28.825701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd884c-5e76-4c78-b8a8-510ab8105fba None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:28.825970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd884c-5e76-4c78-b8a8-510ab8105fba None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:28.826262 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd884c-5e76-4c78-b8a8-510ab8105fba None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:28.826945 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd884c-5e76-4c78-b8a8-510ab8105fba None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:28.827247 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd884c-5e76-4c78-b8a8-510ab8105fba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:28.828662 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd884c-5e76-4c78-b8a8-510ab8105fba None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:28.840333 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 355/716] 10.222.0.20 () {56 vars in 1124 bytes} [Sat Aug 6 12:18:28 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:28.840333 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd884c-5e76-4c78-b8a8-510ab8105fba None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:28.869223 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e21715-25c8-43ef-9aea-079d3231ea00 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:28.939503 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b3900-3670-4398-b956-7db6aa8ae0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:28.941677 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b3900-3670-4398-b956-7db6aa8ae0bc None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:28.941956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b3900-3670-4398-b956-7db6aa8ae0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:28.942227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b3900-3670-4398-b956-7db6aa8ae0bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:28.960874 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b3900-3670-4398-b956-7db6aa8ae0bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:28.963252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9b3900-3670-4398-b956-7db6aa8ae0bc None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:28.979797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-7b9b3900-3670-4398-b956-7db6aa8ae0bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:28.979797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:28.979797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:28.979797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:28.979797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:28.979797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:28.979797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:28.979797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:28.979797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:28.979797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:28.979797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:28.979797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:28.979797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:28.979797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:28.979797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:28.980774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:28.980774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:28.980774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:28.980774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:28.980774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:28.980774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:28.980774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:28.980774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:28.980774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:28.980774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:28.980774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:28.980774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:28.980774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:28.980774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:28.980774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:28.980774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:28.980774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:28.981658 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:28.981658 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:28.981658 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:28.981658 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:28.988274 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 362/717] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:28.990034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b3900-3670-4398-b956-7db6aa8ae0bc None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:29.019370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:29.070220 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd884c-5e76-4c78-b8a8-510ab8105fba None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:29.072443 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:29.116768 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f497a-5ff8-44df-aaac-84d8478e0f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:29.117346 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b3900-3670-4398-b956-7db6aa8ae0bc None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:29.117459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f497a-5ff8-44df-aaac-84d8478e0f37 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:29.118190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f497a-5ff8-44df-aaac-84d8478e0f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:29.118521 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f497a-5ff8-44df-aaac-84d8478e0f37 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:29.130462 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f497a-5ff8-44df-aaac-84d8478e0f37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:29.137307 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-6f7f497a-5ff8-44df-aaac-84d8478e0f37 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:29.137307 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:29.137307 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:29.137307 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:29.137307 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:29.137307 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:29.137307 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:29.137307 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:29.137307 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:29.137307 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:29.137307 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:29.137307 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:29.137307 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:29.137307 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:29.137307 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:29.137307 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:29.138271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:29.138271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:29.138271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:29.138271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:29.138271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:29.138271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:29.138271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:29.138271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:29.138271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:29.138271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:29.138271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:29.138271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:29.138271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:29.138271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:29.138271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:29.138271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:29.139518 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:29.139518 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:29.139518 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:29.139518 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:29.139518 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:29.139518 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:29.139518 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:29.139518 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:29.139518 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:29.139518 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:29.139518 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:29.139518 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:29.139518 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 356/718] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Aug 6 12:18:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:29.149454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f497a-5ff8-44df-aaac-84d8478e0f37 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:29.180835 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09a77c-ae3a-435e-9e21-b9d00ac62aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:29.181514 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09a77c-ae3a-435e-9e21-b9d00ac62aee None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:29.181791 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09a77c-ae3a-435e-9e21-b9d00ac62aee None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:29.182058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09a77c-ae3a-435e-9e21-b9d00ac62aee None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:29.182770 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de09a77c-ae3a-435e-9e21-b9d00ac62aee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:29.183062 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de09a77c-ae3a-435e-9e21-b9d00ac62aee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:29.185708 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de09a77c-ae3a-435e-9e21-b9d00ac62aee None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:29.207856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 363/719] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:29.213804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09a77c-ae3a-435e-9e21-b9d00ac62aee None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:29.281755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f497a-5ff8-44df-aaac-84d8478e0f37 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:29.316036 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:29.327652 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233c035-31e1-4393-a44e-432e160b39d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:29.328644 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233c035-31e1-4393-a44e-432e160b39d6 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:29.329190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233c035-31e1-4393-a44e-432e160b39d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:29.329775 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233c035-31e1-4393-a44e-432e160b39d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:29.330930 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7233c035-31e1-4393-a44e-432e160b39d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:29.331232 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7233c035-31e1-4393-a44e-432e160b39d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:29.333243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7233c035-31e1-4393-a44e-432e160b39d6 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:29.339655 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09a77c-ae3a-435e-9e21-b9d00ac62aee None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:29.344892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 357/720] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Aug 6 12:18:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:29.346260 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb16ddb-a9de-43b7-b35a-6521686b461e None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:29.346542 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb16ddb-a9de-43b7-b35a-6521686b461e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:29.346815 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb16ddb-a9de-43b7-b35a-6521686b461e None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:29.348878 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 358/721] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:29.351227 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb16ddb-a9de-43b7-b35a-6521686b461e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:29.381726 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930fd91-b198-49dc-8c97-93c84ee15e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:29.382502 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930fd91-b198-49dc-8c97-93c84ee15e41 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:29.382842 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930fd91-b198-49dc-8c97-93c84ee15e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:29.383212 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930fd91-b198-49dc-8c97-93c84ee15e41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:29.385207 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:29.391943 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1930fd91-b198-49dc-8c97-93c84ee15e41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:29.394082 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1930fd91-b198-49dc-8c97-93c84ee15e41 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:29.406341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-1930fd91-b198-49dc-8c97-93c84ee15e41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:29.406341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:29.406341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:29.406341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:29.406341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:29.406341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:29.406341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:29.406341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:29.406341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:29.406341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:29.406341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:29.406341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:29.406341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:29.406341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:29.406341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:29.407225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:29.407225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:29.407225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:29.407225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:29.407225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:29.407225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:29.407225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:29.407225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:29.407225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:29.407225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:29.407225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:29.407225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:29.407225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:29.407225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:29.407225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:29.407225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:29.407225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:29.408106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:29.408106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:29.408106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:29.408106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:29.408106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:29.408106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:29.408106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:29.408106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:29.412287 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 364/722] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:29.414036 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44623655-b43f-4e67-8cbf-89a92b5e98c4 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:29.414198 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44623655-b43f-4e67-8cbf-89a92b5e98c4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:29.415122 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44623655-b43f-4e67-8cbf-89a92b5e98c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:29.434496 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-44623655-b43f-4e67-8cbf-89a92b5e98c4 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:29.435870 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-44623655-b43f-4e67-8cbf-89a92b5e98c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:29.487520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb16ddb-a9de-43b7-b35a-6521686b461e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:29.523297 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 365/723] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:18:29.524528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44623655-b43f-4e67-8cbf-89a92b5e98c4 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:29.529929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189c534-714a-4dba-b810-19c7d306917b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:29.530812 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189c534-714a-4dba-b810-19c7d306917b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:29.531285 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189c534-714a-4dba-b810-19c7d306917b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:29.531751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189c534-714a-4dba-b810-19c7d306917b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:29.544922 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c189c534-714a-4dba-b810-19c7d306917b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:29.547140 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c189c534-714a-4dba-b810-19c7d306917b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:29.564893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-c189c534-714a-4dba-b810-19c7d306917b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:29.564893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:29.564893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:29.564893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:29.564893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:29.564893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:29.564893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:29.564893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:29.564893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:29.564893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:29.564893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:29.564893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:29.564893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:29.564893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:29.564893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:29.565825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:29.565825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:29.565825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:29.565825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:29.565825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:29.565825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:29.565825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:29.565825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:29.565825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:29.565825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:29.565825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:29.565825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:29.565825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:29.565825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:29.565825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:29.565825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:29.565825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:29.566794 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:29.566794 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:29.566794 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:29.566794 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:29.567584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 359/724] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:29.569476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189c534-714a-4dba-b810-19c7d306917b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:29.621420 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:29.660536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44623655-b43f-4e67-8cbf-89a92b5e98c4 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:29.669403 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:29.695505 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189c534-714a-4dba-b810-19c7d306917b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:29.705494 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02313522-aebb-4719-9de1-245359839fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:29.706331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02313522-aebb-4719-9de1-245359839fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:29.707025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02313522-aebb-4719-9de1-245359839fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:29.707783 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02313522-aebb-4719-9de1-245359839fe3 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:29.708719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02313522-aebb-4719-9de1-245359839fe3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:29.709165 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02313522-aebb-4719-9de1-245359839fe3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:29.712031 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02313522-aebb-4719-9de1-245359839fe3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:29.727658 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 366/725] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:29.728856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02313522-aebb-4719-9de1-245359839fe3 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:29.737370 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87e8459-dac5-49d0-8e54-4ff288bc4588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:29.738191 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e8459-dac5-49d0-8e54-4ff288bc4588 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:29.738511 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e8459-dac5-49d0-8e54-4ff288bc4588 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:29.738882 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e8459-dac5-49d0-8e54-4ff288bc4588 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:29.751853 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87e8459-dac5-49d0-8e54-4ff288bc4588 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:29.757698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-a87e8459-dac5-49d0-8e54-4ff288bc4588 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:18:29.757698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:29.757698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:29.757698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:29.757698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:29.757698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:29.757698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:29.757698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:29.757698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:29.757698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:29.757698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:29.757698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:29.757698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:29.757698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:29.757698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:29.757698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:29.758977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:29.758977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:29.758977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:29.758977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:29.758977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:29.758977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:29.758977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:29.758977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:29.758977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:29.758977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:29.758977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:29.758977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:29.758977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:29.758977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:29.758977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:29.758977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:29.760035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:29.760035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:29.760035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:29.760035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:29.760035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:29.760035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:29.760035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:29.760035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:29.760035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:29.760035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:29.760035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:18:29.760035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:29.766314 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 360/726] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:18:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:29.767332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87e8459-dac5-49d0-8e54-4ff288bc4588 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:29.852531 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02313522-aebb-4719-9de1-245359839fe3 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:29.883181 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:29.886736 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87e8459-dac5-49d0-8e54-4ff288bc4588 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:29.891362 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9188f-2e83-45ab-8c7c-90f6eeb944fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:29.892055 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9188f-2e83-45ab-8c7c-90f6eeb944fd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:29.892429 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9188f-2e83-45ab-8c7c-90f6eeb944fd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:29.892769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9188f-2e83-45ab-8c7c-90f6eeb944fd None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:29.893534 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9188f-2e83-45ab-8c7c-90f6eeb944fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:29.893910 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9188f-2e83-45ab-8c7c-90f6eeb944fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:29.895336 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d9188f-2e83-45ab-8c7c-90f6eeb944fd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:29.909425 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 367/727] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:29.910399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9188f-2e83-45ab-8c7c-90f6eeb944fd None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:29.929136 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce5db4-21f2-4b0d-841d-198c1d606a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:29.930704 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce5db4-21f2-4b0d-841d-198c1d606a51 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:29.931167 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce5db4-21f2-4b0d-841d-198c1d606a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:29.931860 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce5db4-21f2-4b0d-841d-198c1d606a51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:29.932842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce5db4-21f2-4b0d-841d-198c1d606a51 None admin] RBAC: Adding query filter params (role.id=144c8425688b409384e1e8ed8279d88e, scope.project.id=13fba33c46cf40bbb15a08c690b4c98c, user.id=524b728bf63c4ca88e0f25e531cf5056) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:29.933344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce5db4-21f2-4b0d-841d-198c1d606a51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:29.935372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce5db4-21f2-4b0d-841d-198c1d606a51 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:29.939333 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:29.944461 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 361/728] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:18:29 2022] GET /identity/v3/role_assignments?user.id=524b728bf63c4ca88e0f25e531cf5056&scope.project.id=13fba33c46cf40bbb15a08c690b4c98c&role.id=144c8425688b409384e1e8ed8279d88e => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:29.945495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce5db4-21f2-4b0d-841d-198c1d606a51 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:30.092170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9188f-2e83-45ab-8c7c-90f6eeb944fd None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:30.164436 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68194216-b3dc-4513-8bfa-21adc3e0dbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:30.165979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68194216-b3dc-4513-8bfa-21adc3e0dbaa None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:30.166621 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68194216-b3dc-4513-8bfa-21adc3e0dbaa None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:30.167940 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68194216-b3dc-4513-8bfa-21adc3e0dbaa None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:30.169138 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68194216-b3dc-4513-8bfa-21adc3e0dbaa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:30.170958 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68194216-b3dc-4513-8bfa-21adc3e0dbaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:30.173020 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68194216-b3dc-4513-8bfa-21adc3e0dbaa None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:30.186039 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 368/729] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:18:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:30.186333 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce5db4-21f2-4b0d-841d-198c1d606a51 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:30.192057 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68194216-b3dc-4513-8bfa-21adc3e0dbaa None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:30.247960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7bbe95-97da-4825-8f70-2835acc21345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:30.248677 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bbe95-97da-4825-8f70-2835acc21345 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:30.248909 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bbe95-97da-4825-8f70-2835acc21345 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:30.249181 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bbe95-97da-4825-8f70-2835acc21345 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:30.261203 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7bbe95-97da-4825-8f70-2835acc21345 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:30.262710 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:30.264026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7bbe95-97da-4825-8f70-2835acc21345 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:30.280729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-ca7bbe95-97da-4825-8f70-2835acc21345 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 12:18:30.280729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:30.280729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:30.280729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:30.280729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:30.280729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:30.280729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:30.280729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:30.280729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:30.280729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:30.280729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:30.280729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:30.280729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:30.280729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:30.280729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:30.280729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:30.281721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:30.281721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:30.281721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:30.281721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:30.281721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:30.281721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:30.281721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:30.281721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:30.281721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:30.281721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:30.281721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:30.281721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:30.281721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:30.281721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:30.281721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:30.281721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:30.281721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:30.281721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:30.282755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:30.282755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:30.282755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:30.282755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:30.282755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:30.282755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:30.282755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:30.282755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:30.282755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 12:18:30.282755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:30.282755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 362/730] 10.222.0.20 () {56 vars in 1117 bytes} [Sat Aug 6 12:18:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:30.283888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7caefa1-ed64-4af8-accd-88d8637c1f99 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:30.283888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7caefa1-ed64-4af8-accd-88d8637c1f99 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:30.284053 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7caefa1-ed64-4af8-accd-88d8637c1f99 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:30.285733 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 363/731] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:30.288147 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7caefa1-ed64-4af8-accd-88d8637c1f99 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:30.321376 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68194216-b3dc-4513-8bfa-21adc3e0dbaa None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:30.361676 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e3e4be-d49d-47ea-a9c2-05ae61cbd7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:30.362676 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3e4be-d49d-47ea-a9c2-05ae61cbd7af None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:30.363149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3e4be-d49d-47ea-a9c2-05ae61cbd7af None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:30.363764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3e4be-d49d-47ea-a9c2-05ae61cbd7af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:30.368434 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:30.377969 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e3e4be-d49d-47ea-a9c2-05ae61cbd7af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:30.379713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e3e4be-d49d-47ea-a9c2-05ae61cbd7af None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:30.395720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-52e3e4be-d49d-47ea-a9c2-05ae61cbd7af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:30.395720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:30.395720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:30.395720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:30.395720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:30.395720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:30.395720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:30.395720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:30.395720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:30.395720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:30.395720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:30.395720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:30.395720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:30.395720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:30.395720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:30.396920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:30.396920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:30.396920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:30.396920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:30.396920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:30.396920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:30.396920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:30.396920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:30.396920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:30.396920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:30.396920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:30.396920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:30.396920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:30.396920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:30.396920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:30.396920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:30.396920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:30.397804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:30.397804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:30.397804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:30.397804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:30.404917 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 369/732] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:30.406348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ea478-53a9-4d6e-b379-b8756b83d96f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:30.406912 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ea478-53a9-4d6e-b379-b8756b83d96f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:30.407479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ea478-53a9-4d6e-b379-b8756b83d96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:30.413978 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7caefa1-ed64-4af8-accd-88d8637c1f99 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:30.427592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9d0ea478-53a9-4d6e-b379-b8756b83d96f None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:30.429275 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9d0ea478-53a9-4d6e-b379-b8756b83d96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:30.454866 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2dddf-fbb2-4bd1-855a-744b6de0c53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:30.455626 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2dddf-fbb2-4bd1-855a-744b6de0c53a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:30.456008 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2dddf-fbb2-4bd1-855a-744b6de0c53a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:30.456283 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2dddf-fbb2-4bd1-855a-744b6de0c53a None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:30.457111 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2dddf-fbb2-4bd1-855a-744b6de0c53a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:30.457676 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2dddf-fbb2-4bd1-855a-744b6de0c53a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:30.459539 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2dddf-fbb2-4bd1-855a-744b6de0c53a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:30.481789 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 364/733] 10.222.0.20 () {56 vars in 1149 bytes} [Sat Aug 6 12:18:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:30.483224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2dddf-fbb2-4bd1-855a-744b6de0c53a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:30.512676 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 370/734] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:18:30.513811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ea478-53a9-4d6e-b379-b8756b83d96f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:30.554846 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:30.600293 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2dddf-fbb2-4bd1-855a-744b6de0c53a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:30.624884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:30.629919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ea478-53a9-4d6e-b379-b8756b83d96f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:30.637466 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f5e69-7d8e-43c8-9af6-f1bad50d2e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:30.638081 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f5e69-7d8e-43c8-9af6-f1bad50d2e26 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:30.638358 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f5e69-7d8e-43c8-9af6-f1bad50d2e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:30.638628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f5e69-7d8e-43c8-9af6-f1bad50d2e26 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:30.639313 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f5e69-7d8e-43c8-9af6-f1bad50d2e26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:30.639609 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f5e69-7d8e-43c8-9af6-f1bad50d2e26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:30.641011 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f5e69-7d8e-43c8-9af6-f1bad50d2e26 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:30.657820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 365/735] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:30.657820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f5e69-7d8e-43c8-9af6-f1bad50d2e26 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:30.674839 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1364dd-ef13-4138-9b49-f65cb3482e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:30.675757 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1364dd-ef13-4138-9b49-f65cb3482e11 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:30.676229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1364dd-ef13-4138-9b49-f65cb3482e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:30.676719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1364dd-ef13-4138-9b49-f65cb3482e11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:30.750687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1364dd-ef13-4138-9b49-f65cb3482e11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:30.752513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1364dd-ef13-4138-9b49-f65cb3482e11 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:30.758731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f5e69-7d8e-43c8-9af6-f1bad50d2e26 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:30.787128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:30.793470 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474e3a6-3351-42ed-8445-7455443eeaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:30.794086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474e3a6-3351-42ed-8445-7455443eeaaa None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:30.794362 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474e3a6-3351-42ed-8445-7455443eeaaa None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:30.794634 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474e3a6-3351-42ed-8445-7455443eeaaa None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:30.795331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7474e3a6-3351-42ed-8445-7455443eeaaa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:30.796750 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7474e3a6-3351-42ed-8445-7455443eeaaa None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:30.822474 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 366/736] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Aug 6 12:18:30 2022] POST /identity/v3/services => generated 240 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:18:30.822474 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474e3a6-3351-42ed-8445-7455443eeaaa None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:30.920719 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:30.933103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 371/737] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:18:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:18:30.939748 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1364dd-ef13-4138-9b49-f65cb3482e11 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:31.041440 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474e3a6-3351-42ed-8445-7455443eeaaa None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:31.076206 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9660bfa-72a7-4c4d-af04-b08b42e15205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:31.077125 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9660bfa-72a7-4c4d-af04-b08b42e15205 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:31.077563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9660bfa-72a7-4c4d-af04-b08b42e15205 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:31.078029 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9660bfa-72a7-4c4d-af04-b08b42e15205 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:31.082468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1364dd-ef13-4138-9b49-f65cb3482e11 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:31.086217 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9660bfa-72a7-4c4d-af04-b08b42e15205 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:31.087999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9660bfa-72a7-4c4d-af04-b08b42e15205 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:31.099983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-b9660bfa-72a7-4c4d-af04-b08b42e15205 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 12:18:31.099983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:31.099983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:31.099983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:31.099983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:31.099983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:31.099983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:31.099983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:31.099983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:31.099983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:31.099983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:31.099983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:31.099983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:31.099983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:31.099983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:31.099983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:31.101184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:31.101184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:31.101184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:31.101184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:31.101184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:31.101184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:31.101184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:31.101184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:31.101184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:31.101184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:31.101184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:31.101184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:31.101184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:31.101184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:31.101184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:31.101184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:31.101184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:31.101184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:31.102795 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:31.102795 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:31.102795 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:31.102795 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:31.102795 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:31.102795 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:31.102795 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:31.102795 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:31.102795 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 12:18:31.102795 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:31.104949 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 367/738] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:18:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:31.110888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9660bfa-72a7-4c4d-af04-b08b42e15205 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:31.119382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70270f8-704f-46e8-bee1-255c94bafef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:31.120199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70270f8-704f-46e8-bee1-255c94bafef6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:31.120517 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70270f8-704f-46e8-bee1-255c94bafef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:31.120790 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70270f8-704f-46e8-bee1-255c94bafef6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:31.124021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:31.136271 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70270f8-704f-46e8-bee1-255c94bafef6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:31.137828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70270f8-704f-46e8-bee1-255c94bafef6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:31.156552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-d70270f8-704f-46e8-bee1-255c94bafef6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:31.156552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:31.156552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:31.156552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:31.156552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:31.156552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:31.156552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:31.156552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:31.156552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:31.156552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:31.156552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:31.156552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:31.156552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:31.156552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:31.156552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:31.157509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:31.157509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:31.157509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:31.157509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:31.157509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:31.157509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:31.157509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:31.157509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:31.157509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:31.157509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:31.157509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:31.157509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:31.157509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:31.157509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:31.157509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:31.157509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:31.157509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:31.158397 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:31.158397 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:31.158397 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:31.158397 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:31.162315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 372/739] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:31.163630 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa266a4-5127-4296-8349-4b196b826ca0 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:31.163913 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa266a4-5127-4296-8349-4b196b826ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:31.164191 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa266a4-5127-4296-8349-4b196b826ca0 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:31.166106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 373/740] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:31.167127 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa266a4-5127-4296-8349-4b196b826ca0 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:31.223105 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:31.229782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9660bfa-72a7-4c4d-af04-b08b42e15205 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:31.267004 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7137cfb9-2c19-418f-ac3c-0738a17422f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:31.267563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137cfb9-2c19-418f-ac3c-0738a17422f3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:31.267842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137cfb9-2c19-418f-ac3c-0738a17422f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:31.268108 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137cfb9-2c19-418f-ac3c-0738a17422f3 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:31.268952 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7137cfb9-2c19-418f-ac3c-0738a17422f3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:31.269252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7137cfb9-2c19-418f-ac3c-0738a17422f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:31.270672 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7137cfb9-2c19-418f-ac3c-0738a17422f3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:31.288193 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 368/741] 10.222.0.20 () {56 vars in 1135 bytes} [Sat Aug 6 12:18:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:31.293662 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160be22-7e68-4e18-afcf-c37d30fcc590 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:31.293876 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160be22-7e68-4e18-afcf-c37d30fcc590 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:31.294149 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160be22-7e68-4e18-afcf-c37d30fcc590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:31.297661 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa266a4-5127-4296-8349-4b196b826ca0 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:31.315995 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-0160be22-7e68-4e18-afcf-c37d30fcc590 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:31.317408 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0160be22-7e68-4e18-afcf-c37d30fcc590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:31.338027 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486c04b-dde0-44f3-9291-924caf54fc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:31.339458 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486c04b-dde0-44f3-9291-924caf54fc1c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:31.340238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486c04b-dde0-44f3-9291-924caf54fc1c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:31.340780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486c04b-dde0-44f3-9291-924caf54fc1c None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:31.341665 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5486c04b-dde0-44f3-9291-924caf54fc1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:31.342170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5486c04b-dde0-44f3-9291-924caf54fc1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:31.343881 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5486c04b-dde0-44f3-9291-924caf54fc1c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:31.360977 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 374/742] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:31.361882 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486c04b-dde0-44f3-9291-924caf54fc1c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:31.396594 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 369/743] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:31 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 06 12:18:31.397815 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0160be22-7e68-4e18-afcf-c37d30fcc590 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:31.424516 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:31.493786 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:31.499560 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486c04b-dde0-44f3-9291-924caf54fc1c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:31.506766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0160be22-7e68-4e18-afcf-c37d30fcc590 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:31.543512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743346d-e6c7-4c66-9bb6-c3001f2f2e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:31.544684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743346d-e6c7-4c66-9bb6-c3001f2f2e09 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:31.545175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743346d-e6c7-4c66-9bb6-c3001f2f2e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:31.545655 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743346d-e6c7-4c66-9bb6-c3001f2f2e09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:31.547453 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa5776-54e5-49d0-84ad-57b2cde06be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:31.548260 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa5776-54e5-49d0-84ad-57b2cde06be9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:31.548621 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa5776-54e5-49d0-84ad-57b2cde06be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:31.548901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa5776-54e5-49d0-84ad-57b2cde06be9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:31.554664 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c743346d-e6c7-4c66-9bb6-c3001f2f2e09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:31.556649 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c743346d-e6c7-4c66-9bb6-c3001f2f2e09 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:31.569091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-c743346d-e6c7-4c66-9bb6-c3001f2f2e09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:31.569091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:31.569091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:31.569091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:31.569091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:31.569091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:31.569091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:31.569091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:31.569091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:31.569091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:31.569091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:31.569091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:31.569091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:31.569091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:31.569091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:31.570060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:31.570060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:31.570060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:31.570060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:31.570060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:31.570060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:31.570060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:31.570060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:31.570060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:31.570060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:31.570060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:31.570060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:31.570060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:31.570060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:31.570060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:31.570060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:31.570060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:31.571500 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:31.571500 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:31.571500 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:31.571500 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:31.571500 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:31.571500 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:31.571500 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:31.571500 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:31.574093 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 370/744] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:31.575286 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743346d-e6c7-4c66-9bb6-c3001f2f2e09 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:31.696625 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743346d-e6c7-4c66-9bb6-c3001f2f2e09 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:31.745395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8307f-6d7f-469e-9074-c53a088635c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:31.746392 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8307f-6d7f-469e-9074-c53a088635c5 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:31.746855 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa5776-54e5-49d0-84ad-57b2cde06be9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:31.747076 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8307f-6d7f-469e-9074-c53a088635c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:31.747545 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8307f-6d7f-469e-9074-c53a088635c5 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:31.748333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa5776-54e5-49d0-84ad-57b2cde06be9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:31.748674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee8307f-6d7f-469e-9074-c53a088635c5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:31.750395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee8307f-6d7f-469e-9074-c53a088635c5 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:31.766934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 371/745] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Aug 6 12:18:31 2022] POST /identity/v3/services => generated 251 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:18:31.768188 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8307f-6d7f-469e-9074-c53a088635c5 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:31.804791 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:31.841769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:31.956973 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 375/746] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:18:31 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 06 12:18:31.962889 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa5776-54e5-49d0-84ad-57b2cde06be9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:31.981815 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8307f-6d7f-469e-9074-c53a088635c5 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:32.028150 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d4421-fd23-4876-8a12-ac9cdc1e5b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:32.029159 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d4421-fd23-4876-8a12-ac9cdc1e5b41 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:32.029635 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d4421-fd23-4876-8a12-ac9cdc1e5b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:32.030586 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d4421-fd23-4876-8a12-ac9cdc1e5b41 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:32.031577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d4421-fd23-4876-8a12-ac9cdc1e5b41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:32.032556 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d4421-fd23-4876-8a12-ac9cdc1e5b41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:32.034643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d4421-fd23-4876-8a12-ac9cdc1e5b41 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:32.052168 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 372/747] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:32.059180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d4421-fd23-4876-8a12-ac9cdc1e5b41 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:32.107618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa5776-54e5-49d0-84ad-57b2cde06be9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:32.135961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:32.151595 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a18c19-3ca2-4774-b5e5-160be84b7290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:32.153147 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a18c19-3ca2-4774-b5e5-160be84b7290 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:32.153431 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a18c19-3ca2-4774-b5e5-160be84b7290 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:32.153703 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a18c19-3ca2-4774-b5e5-160be84b7290 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:32.170207 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a18c19-3ca2-4774-b5e5-160be84b7290 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:32.172490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a18c19-3ca2-4774-b5e5-160be84b7290 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:32.182511 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:32.190291 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d4421-fd23-4876-8a12-ac9cdc1e5b41 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:32.194945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-76a18c19-3ca2-4774-b5e5-160be84b7290 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:32.194945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:32.194945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:32.194945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:32.194945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:32.194945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:32.194945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:32.194945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:32.194945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:32.194945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:32.194945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:32.194945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:32.194945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:32.194945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:32.194945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:32.195898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:32.195898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:32.195898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:32.195898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:32.195898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:32.195898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:32.195898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:32.195898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:32.195898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:32.195898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:32.195898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:32.195898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:32.195898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:32.195898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:32.195898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:32.195898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:32.195898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:32.197113 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:32.197113 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:32.197113 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:32.197113 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:32.199500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 376/748] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:32.201404 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e9080-270e-43f9-8766-f97a085389d9 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:32.201903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e9080-270e-43f9-8766-f97a085389d9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:32.202369 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e9080-270e-43f9-8766-f97a085389d9 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:32.204455 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 377/749] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:32.205799 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e9080-270e-43f9-8766-f97a085389d9 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:32.234209 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea0fe7-8a96-4d3e-901b-4b6ea149a0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:32.234848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea0fe7-8a96-4d3e-901b-4b6ea149a0bc None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:32.235219 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea0fe7-8a96-4d3e-901b-4b6ea149a0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:32.235492 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea0fe7-8a96-4d3e-901b-4b6ea149a0bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:32.236669 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea0fe7-8a96-4d3e-901b-4b6ea149a0bc None admin] RBAC: Adding query filter params (role.id=785cca921bad4984a86e45ce6c5601c2, scope.project.id=13fba33c46cf40bbb15a08c690b4c98c, user.id=c810b52785b94c0cab4fd9f0d6e65825) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:32.237024 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea0fe7-8a96-4d3e-901b-4b6ea149a0bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:32.239200 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea0fe7-8a96-4d3e-901b-4b6ea149a0bc None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:32.252330 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 373/750] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:18:32 2022] GET /identity/v3/role_assignments?user.id=c810b52785b94c0cab4fd9f0d6e65825&scope.project.id=13fba33c46cf40bbb15a08c690b4c98c&role.id=785cca921bad4984a86e45ce6c5601c2 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:32.252330 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c6b3d-591d-4c1b-93df-cf7b56dd84bf None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:32.252330 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c6b3d-591d-4c1b-93df-cf7b56dd84bf None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:32.252330 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c6b3d-591d-4c1b-93df-cf7b56dd84bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:32.276679 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-dc8c6b3d-591d-4c1b-93df-cf7b56dd84bf None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:32.281295 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-dc8c6b3d-591d-4c1b-93df-cf7b56dd84bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:32.349113 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e9080-270e-43f9-8766-f97a085389d9 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:32.389596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50942e6e-3d62-429b-9fba-d153c0016c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:32.390228 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50942e6e-3d62-429b-9fba-d153c0016c06 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:32.390498 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50942e6e-3d62-429b-9fba-d153c0016c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:32.390769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50942e6e-3d62-429b-9fba-d153c0016c06 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:32.391466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50942e6e-3d62-429b-9fba-d153c0016c06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:32.391769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50942e6e-3d62-429b-9fba-d153c0016c06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:32.393199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50942e6e-3d62-429b-9fba-d153c0016c06 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:32.406560 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 378/751] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:32.412070 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50942e6e-3d62-429b-9fba-d153c0016c06 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:32.425476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 374/752] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:18:32.450897 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c6b3d-591d-4c1b-93df-cf7b56dd84bf None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:32.526289 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:32.548728 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:32.606075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50942e6e-3d62-429b-9fba-d153c0016c06 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:32.637490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c6b3d-591d-4c1b-93df-cf7b56dd84bf None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:32.677027 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fcf8a6-11c4-4441-97e0-a625996e1f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:32.677681 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcf8a6-11c4-4441-97e0-a625996e1f76 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:32.677958 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcf8a6-11c4-4441-97e0-a625996e1f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:32.678227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcf8a6-11c4-4441-97e0-a625996e1f76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:32.687636 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fcf8a6-11c4-4441-97e0-a625996e1f76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:32.689192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fcf8a6-11c4-4441-97e0-a625996e1f76 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:32.702391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-74fcf8a6-11c4-4441-97e0-a625996e1f76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:32.702391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:32.702391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:32.702391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:32.702391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:32.702391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:32.702391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:32.702391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:32.702391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:32.702391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:32.702391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:32.702391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:32.702391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:32.702391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:32.702391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:32.703321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:32.703321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:32.703321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:32.703321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:32.703321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:32.703321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:32.703321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:32.703321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:32.703321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:32.703321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:32.703321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:32.703321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:32.703321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:32.703321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:32.703321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:32.703321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:32.703321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:32.704219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:32.704219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:32.704219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:32.704219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:32.704219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:32.704219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:32.704219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:32.704219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:32.704219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 379/753] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:32.711243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1c0ba-fc65-4917-a73a-6e0c2d214c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:32.711738 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fcf8a6-11c4-4441-97e0-a625996e1f76 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:32.711950 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1c0ba-fc65-4917-a73a-6e0c2d214c85 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:32.712322 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1c0ba-fc65-4917-a73a-6e0c2d214c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:32.712613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1c0ba-fc65-4917-a73a-6e0c2d214c85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:32.723532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1c0ba-fc65-4917-a73a-6e0c2d214c85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:32.728507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-c4c1c0ba-fc65-4917-a73a-6e0c2d214c85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:18:32.728507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:32.728507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:32.728507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:32.728507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:32.728507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:32.728507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:32.728507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:32.728507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:32.728507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:32.728507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:32.728507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:32.728507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:32.728507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:32.728507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:32.728507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:32.730348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:32.730348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:32.730348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:32.730348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:32.730348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:32.730348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:32.730348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:32.730348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:32.730348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:32.730348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:32.730348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:32.730348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:32.730348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:32.730348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:32.730348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:32.730348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:32.731373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:32.731373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:32.731373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:32.731373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:32.731373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:32.731373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:32.731373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:32.731373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:32.731373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:32.731373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:32.731373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:18:32.731373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:32.731373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 375/754] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:18:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:32.749015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1c0ba-fc65-4917-a73a-6e0c2d214c85 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:32.830176 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fcf8a6-11c4-4441-97e0-a625996e1f76 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:32.848178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:32.875742 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3e9a8-cd2e-445c-b94a-5e8832de6b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:32.876282 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3e9a8-cd2e-445c-b94a-5e8832de6b45 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:32.876580 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3e9a8-cd2e-445c-b94a-5e8832de6b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:32.876852 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3e9a8-cd2e-445c-b94a-5e8832de6b45 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:32.877612 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3e9a8-cd2e-445c-b94a-5e8832de6b45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:32.877921 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3e9a8-cd2e-445c-b94a-5e8832de6b45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:32.879346 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3e9a8-cd2e-445c-b94a-5e8832de6b45 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:32.900411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1c0ba-fc65-4917-a73a-6e0c2d214c85 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:32.902973 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 380/755] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:32.909865 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3e9a8-cd2e-445c-b94a-5e8832de6b45 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:32.940649 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:32.985529 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f16cff-5a6b-41ec-86e9-be31f59e3ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:32.986144 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f16cff-5a6b-41ec-86e9-be31f59e3ecd None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:32.986465 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f16cff-5a6b-41ec-86e9-be31f59e3ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:32.986744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f16cff-5a6b-41ec-86e9-be31f59e3ecd None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:32.987458 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f16cff-5a6b-41ec-86e9-be31f59e3ecd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:32.987756 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f16cff-5a6b-41ec-86e9-be31f59e3ecd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:32.989153 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f16cff-5a6b-41ec-86e9-be31f59e3ecd None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:33.008569 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 376/756] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:18:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:33.015514 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f16cff-5a6b-41ec-86e9-be31f59e3ecd None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:33.033620 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3e9a8-cd2e-445c-b94a-5e8832de6b45 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:33.070210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394d97f-3a16-4f16-b8a7-2d9484233a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:33.071722 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394d97f-3a16-4f16-b8a7-2d9484233a7e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:33.072199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394d97f-3a16-4f16-b8a7-2d9484233a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:33.072771 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394d97f-3a16-4f16-b8a7-2d9484233a7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:33.073793 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394d97f-3a16-4f16-b8a7-2d9484233a7e None admin] RBAC: Adding query filter params (role.id=144c8425688b409384e1e8ed8279d88e, scope.project.id=13fba33c46cf40bbb15a08c690b4c98c, user.id=2efc792a2655439f8b8070010059b6bb) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:33.074313 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394d97f-3a16-4f16-b8a7-2d9484233a7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:33.075942 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394d97f-3a16-4f16-b8a7-2d9484233a7e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:33.082432 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 381/757] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:18:32 2022] GET /identity/v3/role_assignments?user.id=2efc792a2655439f8b8070010059b6bb&scope.project.id=13fba33c46cf40bbb15a08c690b4c98c&role.id=144c8425688b409384e1e8ed8279d88e => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:33.110547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da824f1-7817-4e49-8674-b9173f32e549 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:33.110900 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da824f1-7817-4e49-8674-b9173f32e549 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:33.111100 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da824f1-7817-4e49-8674-b9173f32e549 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:33.113593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 382/758] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:33.121871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f41540-4cd3-46c1-b160-d30b01b18166 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:33.122456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f41540-4cd3-46c1-b160-d30b01b18166 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:33.122742 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f41540-4cd3-46c1-b160-d30b01b18166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:33.145190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-53f41540-4cd3-46c1-b160-d30b01b18166 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:33.146907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-53f41540-4cd3-46c1-b160-d30b01b18166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:33.191799 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:33.194663 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f16cff-5a6b-41ec-86e9-be31f59e3ecd None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:33.246152 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 383/759] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 12:18:33.269592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f41540-4cd3-46c1-b160-d30b01b18166 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:33.272592 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3effc28-8aee-4802-b776-11f130e2da1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:33.274202 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3effc28-8aee-4802-b776-11f130e2da1f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:33.274202 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3effc28-8aee-4802-b776-11f130e2da1f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:33.274202 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3effc28-8aee-4802-b776-11f130e2da1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:33.290709 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3effc28-8aee-4802-b776-11f130e2da1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:33.292358 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3effc28-8aee-4802-b776-11f130e2da1f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:33.310725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-b3effc28-8aee-4802-b776-11f130e2da1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:33.310725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:33.310725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:33.310725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:33.310725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:33.310725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:33.310725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:33.310725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:33.310725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:33.310725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:33.310725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:33.310725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:33.310725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:33.310725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:33.310725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:33.311654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:33.311654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:33.311654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:33.311654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:33.311654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:33.311654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:33.311654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:33.311654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:33.311654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:33.311654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:33.311654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:33.311654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:33.311654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:33.311654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:33.311654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:33.311654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:33.311654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:33.318928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:33.318928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:33.318928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:33.318928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:33.318928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 377/760] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:33.319732 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3effc28-8aee-4802-b776-11f130e2da1f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:33.379388 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:33.458543 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f41540-4cd3-46c1-b160-d30b01b18166 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:33.466338 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3effc28-8aee-4802-b776-11f130e2da1f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:33.517542 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca94b1-5246-4be2-9c05-476bebf12341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:33.518441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca94b1-5246-4be2-9c05-476bebf12341 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:33.519718 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca94b1-5246-4be2-9c05-476bebf12341 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:33.521214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca94b1-5246-4be2-9c05-476bebf12341 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:33.522098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bca94b1-5246-4be2-9c05-476bebf12341 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:33.523257 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bca94b1-5246-4be2-9c05-476bebf12341 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:33.525678 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bca94b1-5246-4be2-9c05-476bebf12341 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:33.537562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd63f8-3c1d-445f-9b2a-4b85760e8391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:33.538318 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd63f8-3c1d-445f-9b2a-4b85760e8391 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:33.538485 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd63f8-3c1d-445f-9b2a-4b85760e8391 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:33.538809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd63f8-3c1d-445f-9b2a-4b85760e8391 None admin] PATH_INFO: `/v3/services/network` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:33.538961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 378/761] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:33.539409 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bd63f8-3c1d-445f-9b2a-4b85760e8391 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:33.541294 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bd63f8-3c1d-445f-9b2a-4b85760e8391 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:33.549301 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca94b1-5246-4be2-9c05-476bebf12341 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:33.550762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-51bd63f8-3c1d-445f-9b2a-4b85760e8391 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 12:18:33.550762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:33.550762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:18:33.550762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:18:33.550762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:18:33.550762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:18:33.550762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:18:33.550762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:33.550762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 12:18:33.550762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:33.550762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:18:33.550762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:33.550762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:33.550762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:33.550762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:33.550762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:33.550762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:33.551716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:33.551716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:33.551716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:33.551716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:33.551716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:33.551716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:33.551716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:18:33.551716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:18:33.551716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:18:33.551716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:18:33.551716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:33.551716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:33.551716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:33.551716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:33.551716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:33.551716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:33.552593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:33.552593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:33.552593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:33.552593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:33.552593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:33.552593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:33.552593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:33.552593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:33.552593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:33.552593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:33.552593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:18:33.552593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:33.552593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 12:18:33.552593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:33.554885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 384/762] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:33 2022] GET /identity/v3/services/network => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:33.568124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd63f8-3c1d-445f-9b2a-4b85760e8391 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:33.636109 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:33.721622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bd63f8-3c1d-445f-9b2a-4b85760e8391 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:33.750172 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:33.768643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca94b1-5246-4be2-9c05-476bebf12341 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:33.792378 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5882fe-87b8-4566-97e2-9f9e1d55c118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:33.793001 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5882fe-87b8-4566-97e2-9f9e1d55c118 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:33.793274 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5882fe-87b8-4566-97e2-9f9e1d55c118 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:33.793542 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5882fe-87b8-4566-97e2-9f9e1d55c118 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:33.794217 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5882fe-87b8-4566-97e2-9f9e1d55c118 None admin] RBAC: Adding query filter params (name=network) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:33.794511 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5882fe-87b8-4566-97e2-9f9e1d55c118 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:33.795851 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5882fe-87b8-4566-97e2-9f9e1d55c118 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:33.806626 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 385/763] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Aug 6 12:18:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:33.814637 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5882fe-87b8-4566-97e2-9f9e1d55c118 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:33.826223 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1445f81-e31c-4a42-8570-e9a82b391615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:33.826866 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1445f81-e31c-4a42-8570-e9a82b391615 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:33.827146 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1445f81-e31c-4a42-8570-e9a82b391615 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:33.827420 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1445f81-e31c-4a42-8570-e9a82b391615 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:33.835048 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1445f81-e31c-4a42-8570-e9a82b391615 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:33.836643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1445f81-e31c-4a42-8570-e9a82b391615 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:33.848888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-d1445f81-e31c-4a42-8570-e9a82b391615 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 12:18:33.848888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:33.848888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:33.848888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:33.848888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:33.848888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:33.848888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:33.848888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:33.848888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:33.848888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:33.848888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:33.848888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:33.848888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:33.848888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:33.848888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:33.848888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:33.849828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:33.849828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:33.849828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:33.849828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:33.849828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:33.849828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:33.849828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:33.849828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:33.849828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:33.849828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:33.849828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:33.849828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:33.849828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:33.849828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:33.849828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:33.849828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:33.849828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:33.849828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:33.850760 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:33.850760 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:33.850760 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:33.850760 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:33.850760 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:33.850760 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:33.850760 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:33.850760 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:33.850760 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 12:18:33.850760 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:33.850760 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 379/764] 10.222.0.20 () {56 vars in 1093 bytes} [Sat Aug 6 12:18:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:33.857230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1445f81-e31c-4a42-8570-e9a82b391615 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:33.991854 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5882fe-87b8-4566-97e2-9f9e1d55c118 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:34.002615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:34.032811 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1445f81-e31c-4a42-8570-e9a82b391615 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:34.055368 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d4f92-60a1-4a7c-a92d-b361fa35e394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:34.056117 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d4f92-60a1-4a7c-a92d-b361fa35e394 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:34.056420 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d4f92-60a1-4a7c-a92d-b361fa35e394 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:34.056700 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d4f92-60a1-4a7c-a92d-b361fa35e394 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:34.058273 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d4f92-60a1-4a7c-a92d-b361fa35e394 None admin] RBAC: Adding query filter params (type=network) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:34.058581 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d4f92-60a1-4a7c-a92d-b361fa35e394 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:34.059944 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d4f92-60a1-4a7c-a92d-b361fa35e394 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:34.072349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 386/765] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Aug 6 12:18:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:34.076633 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d4f92-60a1-4a7c-a92d-b361fa35e394 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:34.098861 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82947a-08d6-464e-a465-e94485ee39a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:34.099538 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82947a-08d6-464e-a465-e94485ee39a8 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:34.099823 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82947a-08d6-464e-a465-e94485ee39a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:34.100097 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82947a-08d6-464e-a465-e94485ee39a8 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:34.100931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82947a-08d6-464e-a465-e94485ee39a8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:34.101241 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82947a-08d6-464e-a465-e94485ee39a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:34.103216 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82947a-08d6-464e-a465-e94485ee39a8 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:34.136806 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 380/766] 10.222.0.20 () {56 vars in 1133 bytes} [Sat Aug 6 12:18:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:34.142107 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82947a-08d6-464e-a465-e94485ee39a8 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:34.147332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:34.260078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d4f92-60a1-4a7c-a92d-b361fa35e394 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:34.323854 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e6884-bbc7-4756-b5c1-350e5678b734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:34.324520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e6884-bbc7-4756-b5c1-350e5678b734 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:34.324802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e6884-bbc7-4756-b5c1-350e5678b734 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:34.325073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e6884-bbc7-4756-b5c1-350e5678b734 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:34.325517 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82947a-08d6-464e-a465-e94485ee39a8 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:34.325838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e6884-bbc7-4756-b5c1-350e5678b734 None admin] RBAC: Adding query filter params (interface=public, service_id=ed9f74fd26c740f99f293a41998dd98e, region_id=RegionOne) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:34.326144 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e6884-bbc7-4756-b5c1-350e5678b734 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:34.327516 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e6884-bbc7-4756-b5c1-350e5678b734 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:34.353279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 387/767] 10.222.0.20 () {56 vars in 1251 bytes} [Sat Aug 6 12:18:34 2022] GET /identity/v3/endpoints?service_id=ed9f74fd26c740f99f293a41998dd98e&interface=public®ion_id=RegionOne => generated 194 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:34.355343 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2897582-8f31-4aca-b9a1-4ef0eb96d28f None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:34.355630 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2897582-8f31-4aca-b9a1-4ef0eb96d28f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:34.356165 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2897582-8f31-4aca-b9a1-4ef0eb96d28f None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:34.358325 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 388/768] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:34.361354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2897582-8f31-4aca-b9a1-4ef0eb96d28f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:34.363098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f2ea2-22e4-4d87-8a0c-5cce697b851b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:34.363752 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f2ea2-22e4-4d87-8a0c-5cce697b851b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:34.364040 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f2ea2-22e4-4d87-8a0c-5cce697b851b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:34.364477 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f2ea2-22e4-4d87-8a0c-5cce697b851b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:34.376228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f2ea2-22e4-4d87-8a0c-5cce697b851b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:34.377964 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f2ea2-22e4-4d87-8a0c-5cce697b851b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:34.391233 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:34.395817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-d79f2ea2-22e4-4d87-8a0c-5cce697b851b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:34.395817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:34.395817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:34.395817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:34.395817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:34.395817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:34.395817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:34.395817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:34.395817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:34.395817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:34.395817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:34.395817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:34.395817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:34.395817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:34.395817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:34.396714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:34.396714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:34.396714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:34.396714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:34.396714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:34.396714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:34.396714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:34.396714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:34.396714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:34.396714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:34.396714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:34.396714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:34.396714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:34.396714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:34.396714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:34.396714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:34.396714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:34.397610 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:34.397610 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:34.397610 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:34.397610 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:34.397610 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 381/769] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:34.401946 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e5496-9288-4991-842e-63345c597984 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:34.401946 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e5496-9288-4991-842e-63345c597984 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:34.401946 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e5496-9288-4991-842e-63345c597984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:34.420088 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-177e5496-9288-4991-842e-63345c597984 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:34.421377 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-177e5496-9288-4991-842e-63345c597984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:34.478414 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:34.494115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2897582-8f31-4aca-b9a1-4ef0eb96d28f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:34.504531 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 382/770] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:18:34.505446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e5496-9288-4991-842e-63345c597984 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:34.533384 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e20ce-57db-4330-a114-5514ca01db5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:34.534600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e20ce-57db-4330-a114-5514ca01db5c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:34.535558 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e20ce-57db-4330-a114-5514ca01db5c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:34.536193 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e20ce-57db-4330-a114-5514ca01db5c None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:34.537070 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e20ce-57db-4330-a114-5514ca01db5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:34.539045 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e20ce-57db-4330-a114-5514ca01db5c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:34.547388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 389/771] 10.222.0.20 () {56 vars in 1087 bytes} [Sat Aug 6 12:18:34 2022] GET /identity/v3/services => generated 772 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:34.548771 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e20ce-57db-4330-a114-5514ca01db5c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:34.661112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e5496-9288-4991-842e-63345c597984 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:34.665795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:34.676575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e20ce-57db-4330-a114-5514ca01db5c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:34.714096 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df1aba-ee32-4fa1-add3-b951828634ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:34.715120 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df1aba-ee32-4fa1-add3-b951828634ad None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:34.715606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df1aba-ee32-4fa1-add3-b951828634ad None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:34.716074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df1aba-ee32-4fa1-add3-b951828634ad None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:34.716269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306de4df-760c-48e3-a497-2d2c7a2a1e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:34.716988 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306de4df-760c-48e3-a497-2d2c7a2a1e56 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:34.717268 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306de4df-760c-48e3-a497-2d2c7a2a1e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:34.717542 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306de4df-760c-48e3-a497-2d2c7a2a1e56 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:34.717837 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9df1aba-ee32-4fa1-add3-b951828634ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:34.718183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306de4df-760c-48e3-a497-2d2c7a2a1e56 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:34.718532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9df1aba-ee32-4fa1-add3-b951828634ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:34.719575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306de4df-760c-48e3-a497-2d2c7a2a1e56 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:34.720372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9df1aba-ee32-4fa1-add3-b951828634ad None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:34.730242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-306de4df-760c-48e3-a497-2d2c7a2a1e56 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 12:18:34.730242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:34.730242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:18:34.730242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:18:34.730242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:18:34.730242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:18:34.730242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:18:34.730242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:34.730242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 12:18:34.730242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:34.730242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:18:34.730242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:34.730242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:34.730242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:34.730242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:34.730242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:34.731256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:34.731256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:34.731256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:34.731256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:34.731256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:34.731256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:34.731256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:34.731256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:18:34.731256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:18:34.731256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:18:34.731256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:18:34.731256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:34.731256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:34.731256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:34.731256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:34.731256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:34.732073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:34.732073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:34.732073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:34.732073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:34.732073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:34.732073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:34.732073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:34.732073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:34.732073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:34.732073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:34.732073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:34.732073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:18:34.732073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:34.732073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 12:18:34.732073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:34.732073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 390/772] 10.222.0.20 () {56 vars in 1129 bytes} [Sat Aug 6 12:18:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:34.738773 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306de4df-760c-48e3-a497-2d2c7a2a1e56 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:34.748346 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 383/773] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:34.753488 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df1aba-ee32-4fa1-add3-b951828634ad None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:34.814346 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:34.881118 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306de4df-760c-48e3-a497-2d2c7a2a1e56 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:34.882366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df1aba-ee32-4fa1-add3-b951828634ad None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:34.916869 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed087d0b-aa54-4149-9de5-f46110ef08aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:34.917595 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed087d0b-aa54-4149-9de5-f46110ef08aa None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:34.917945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed087d0b-aa54-4149-9de5-f46110ef08aa None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:34.918285 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed087d0b-aa54-4149-9de5-f46110ef08aa None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:34.919031 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed087d0b-aa54-4149-9de5-f46110ef08aa None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:34.919400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed087d0b-aa54-4149-9de5-f46110ef08aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:34.920860 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed087d0b-aa54-4149-9de5-f46110ef08aa None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:34.925856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 391/774] 10.222.0.20 () {56 vars in 1124 bytes} [Sat Aug 6 12:18:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:34.926977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f2c60-435b-4047-b400-2f855afbedad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:34.927324 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd4aa5-3b0a-40d2-9cd7-d7e830595342 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:34.927617 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f2c60-435b-4047-b400-2f855afbedad None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:34.927725 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd4aa5-3b0a-40d2-9cd7-d7e830595342 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:34.927895 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f2c60-435b-4047-b400-2f855afbedad None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:34.928134 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd4aa5-3b0a-40d2-9cd7-d7e830595342 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:34.928243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f2c60-435b-4047-b400-2f855afbedad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:34.929893 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 392/775] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:34.935167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd4aa5-3b0a-40d2-9cd7-d7e830595342 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:34.936119 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657f2c60-435b-4047-b400-2f855afbedad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:34.937590 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657f2c60-435b-4047-b400-2f855afbedad None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:34.953421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-657f2c60-435b-4047-b400-2f855afbedad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:34.953421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:34.953421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:34.953421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:34.953421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:34.953421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:34.953421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:34.953421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:34.953421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:34.953421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:34.953421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:34.953421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:34.953421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:34.953421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:34.953421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:34.954352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:34.954352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:34.954352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:34.954352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:34.954352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:34.954352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:34.954352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:34.954352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:34.954352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:34.954352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:34.954352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:34.954352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:34.954352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:34.954352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:34.954352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:34.954352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:34.954352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:34.955697 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:34.955697 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:34.955697 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:34.955697 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:34.955697 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:34.955697 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:34.955697 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:34.955697 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:34.955697 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 384/776] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:34.956588 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33bbe9-6062-40a6-9be4-0bd7b603a8a9 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:34.956696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33bbe9-6062-40a6-9be4-0bd7b603a8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:34.958981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33bbe9-6062-40a6-9be4-0bd7b603a8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:34.965747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:34.985147 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-5f33bbe9-6062-40a6-9be4-0bd7b603a8a9 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:34.986619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5f33bbe9-6062-40a6-9be4-0bd7b603a8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:35.069697 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 385/777] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:18:35.070707 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33bbe9-6062-40a6-9be4-0bd7b603a8a9 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:35.077018 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd4aa5-3b0a-40d2-9cd7-d7e830595342 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:35.122494 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e894cd0-75ee-45ae-8abc-a987cb9348d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:35.123222 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e894cd0-75ee-45ae-8abc-a987cb9348d8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:35.123565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e894cd0-75ee-45ae-8abc-a987cb9348d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:35.123904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e894cd0-75ee-45ae-8abc-a987cb9348d8 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:35.124679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e894cd0-75ee-45ae-8abc-a987cb9348d8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:35.125040 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e894cd0-75ee-45ae-8abc-a987cb9348d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:35.126825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e894cd0-75ee-45ae-8abc-a987cb9348d8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:35.126997 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:35.134884 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 393/778] 10.222.0.20 () {56 vars in 1124 bytes} [Sat Aug 6 12:18:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:35.135723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e894cd0-75ee-45ae-8abc-a987cb9348d8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:35.199947 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33bbe9-6062-40a6-9be4-0bd7b603a8a9 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:35.242074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97941a11-a122-4009-bca4-1c4fa2fe4633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:35.243383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97941a11-a122-4009-bca4-1c4fa2fe4633 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:35.243759 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97941a11-a122-4009-bca4-1c4fa2fe4633 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:35.244204 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:35.244579 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97941a11-a122-4009-bca4-1c4fa2fe4633 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:35.245545 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97941a11-a122-4009-bca4-1c4fa2fe4633 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:35.246012 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97941a11-a122-4009-bca4-1c4fa2fe4633 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:35.247979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97941a11-a122-4009-bca4-1c4fa2fe4633 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:35.260872 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e894cd0-75ee-45ae-8abc-a987cb9348d8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:35.268254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 386/779] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:35.269209 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97941a11-a122-4009-bca4-1c4fa2fe4633 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:35.309846 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef9c41-ed4a-404b-af9f-df821b3fdefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:35.310648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef9c41-ed4a-404b-af9f-df821b3fdefe None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:35.310928 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef9c41-ed4a-404b-af9f-df821b3fdefe None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:35.311257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef9c41-ed4a-404b-af9f-df821b3fdefe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:35.326730 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef9c41-ed4a-404b-af9f-df821b3fdefe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:35.333754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-41ef9c41-ed4a-404b-af9f-df821b3fdefe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:35.333754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:35.333754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:35.333754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:35.333754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:35.333754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:35.333754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:35.333754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:35.333754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:35.333754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:35.333754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:35.333754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:35.333754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:35.333754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:35.333754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:35.333754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:35.334729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:35.334729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:35.334729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:35.334729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:35.334729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:35.334729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:35.334729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:35.334729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:35.334729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:35.334729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:35.334729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:35.334729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:35.334729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:35.334729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:35.334729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:35.334729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:35.335565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:35.335565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:35.335565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:35.335565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:35.335565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:35.335565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:35.335565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:35.335565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:35.335565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:35.335565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:35.335565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:35.335565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:35.336999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 394/780] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Aug 6 12:18:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:35.336999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef9c41-ed4a-404b-af9f-df821b3fdefe None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:35.382656 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:35.390320 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97941a11-a122-4009-bca4-1c4fa2fe4633 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:35.425746 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283662e-83ad-4724-8281-ce43a35fba75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:35.426363 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283662e-83ad-4724-8281-ce43a35fba75 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:35.426936 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283662e-83ad-4724-8281-ce43a35fba75 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:35.427227 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283662e-83ad-4724-8281-ce43a35fba75 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:35.427988 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283662e-83ad-4724-8281-ce43a35fba75 None admin] RBAC: Adding query filter params (interface=public, service_id=97ae999b0c7a41a09bf300fd1f7753ad, region_id=RegionOne) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:35.428311 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283662e-83ad-4724-8281-ce43a35fba75 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:35.429683 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283662e-83ad-4724-8281-ce43a35fba75 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:35.441275 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 387/781] 10.222.0.20 () {56 vars in 1251 bytes} [Sat Aug 6 12:18:35 2022] GET /identity/v3/endpoints?service_id=97ae999b0c7a41a09bf300fd1f7753ad&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:35.441275 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283662e-83ad-4724-8281-ce43a35fba75 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:35.453305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef9c41-ed4a-404b-af9f-df821b3fdefe None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:35.487998 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10789c-87f8-4e82-877e-e1a10677fc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:35.488685 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10789c-87f8-4e82-877e-e1a10677fc1c None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:35.488963 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10789c-87f8-4e82-877e-e1a10677fc1c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:35.489239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10789c-87f8-4e82-877e-e1a10677fc1c None admin] PATH_INFO: `/v3/projects/13fba33c46cf40bbb15a08c690b4c98c/users/524b728bf63c4ca88e0f25e531cf5056/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:35.491449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10789c-87f8-4e82-877e-e1a10677fc1c None admin] RBAC: Authorizing `identity:create_grant(project_id=13fba33c46cf40bbb15a08c690b4c98c, user_id=524b728bf63c4ca88e0f25e531cf5056, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:35.492945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10789c-87f8-4e82-877e-e1a10677fc1c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:35.509773 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 395/782] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:18:35 2022] PUT /identity/v3/projects/13fba33c46cf40bbb15a08c690b4c98c/users/524b728bf63c4ca88e0f25e531cf5056/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:18:35.510946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe653b7-cb92-49f1-ba6d-f3cd0001ffc3 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:35.511230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe653b7-cb92-49f1-ba6d-f3cd0001ffc3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:35.511507 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe653b7-cb92-49f1-ba6d-f3cd0001ffc3 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:35.512957 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 396/783] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:35.517808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe653b7-cb92-49f1-ba6d-f3cd0001ffc3 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:35.521836 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:35.566136 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283662e-83ad-4724-8281-ce43a35fba75 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:35.605945 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd4a72-3614-4dd3-8082-ff2d5cb62775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:35.606946 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd4a72-3614-4dd3-8082-ff2d5cb62775 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:35.607370 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd4a72-3614-4dd3-8082-ff2d5cb62775 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:35.607822 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd4a72-3614-4dd3-8082-ff2d5cb62775 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:35.608791 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dd4a72-3614-4dd3-8082-ff2d5cb62775 None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:35.609242 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dd4a72-3614-4dd3-8082-ff2d5cb62775 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:35.611051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dd4a72-3614-4dd3-8082-ff2d5cb62775 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:35.621621 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 388/784] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Aug 6 12:18:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:35.623078 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf814c7-bd79-4c37-88e9-057b4687f8c6 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:35.623362 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf814c7-bd79-4c37-88e9-057b4687f8c6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:35.623643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf814c7-bd79-4c37-88e9-057b4687f8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:35.646051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-2bf814c7-bd79-4c37-88e9-057b4687f8c6 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:35.647311 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2bf814c7-bd79-4c37-88e9-057b4687f8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:35.658907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe653b7-cb92-49f1-ba6d-f3cd0001ffc3 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:35.668519 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:35.711738 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27e99f-96a2-406f-a70e-98cae7012a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:35.712902 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27e99f-96a2-406f-a70e-98cae7012a65 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:35.713705 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27e99f-96a2-406f-a70e-98cae7012a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:35.714179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27e99f-96a2-406f-a70e-98cae7012a65 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:35.715052 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27e99f-96a2-406f-a70e-98cae7012a65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:35.717339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27e99f-96a2-406f-a70e-98cae7012a65 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:35.726659 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 397/785] 10.222.0.20 () {56 vars in 1087 bytes} [Sat Aug 6 12:18:35 2022] GET /identity/v3/services => generated 772 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:35.727938 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27e99f-96a2-406f-a70e-98cae7012a65 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:35.736684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 389/786] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:18:35.760004 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf814c7-bd79-4c37-88e9-057b4687f8c6 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:35.852733 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:35.918838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27e99f-96a2-406f-a70e-98cae7012a65 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:35.953226 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf814c7-bd79-4c37-88e9-057b4687f8c6 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:35.959670 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ad6ee-512c-4ea2-a72b-1b2653c0189c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:35.960346 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ad6ee-512c-4ea2-a72b-1b2653c0189c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:35.960640 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ad6ee-512c-4ea2-a72b-1b2653c0189c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:35.960914 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ad6ee-512c-4ea2-a72b-1b2653c0189c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:35.972512 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6ad6ee-512c-4ea2-a72b-1b2653c0189c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:35.973961 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6ad6ee-512c-4ea2-a72b-1b2653c0189c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:35.990343 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4485601-7058-4976-b8a6-e3e4dee57f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:35.991037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4485601-7058-4976-b8a6-e3e4dee57f10 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:35.991587 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4485601-7058-4976-b8a6-e3e4dee57f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:35.991861 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4485601-7058-4976-b8a6-e3e4dee57f10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:36.001026 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-9c6ad6ee-512c-4ea2-a72b-1b2653c0189c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:36.001026 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:36.001026 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:36.001026 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:36.001026 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:36.001026 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:36.001026 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:36.001026 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:36.001026 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:36.001026 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:36.001026 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:36.001026 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:36.001026 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:36.001026 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:36.001026 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:36.002075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:36.002075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:36.002075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:36.002075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:36.002075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:36.002075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:36.002075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:36.002075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:36.002075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:36.002075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:36.002075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:36.002075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:36.002075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:36.002075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:36.002075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:36.002075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:36.002075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:36.003017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:36.003017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:36.003017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:36.003017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:36.004256 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4485601-7058-4976-b8a6-e3e4dee57f10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:36.005176 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 398/787] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:36.010808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-f4485601-7058-4976-b8a6-e3e4dee57f10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:18:36.010808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:36.010808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:36.010808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:36.010808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:36.010808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:36.010808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:36.010808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:36.010808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:36.010808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:36.010808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:36.010808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:36.010808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:36.010808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:36.010808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:36.010808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:36.011920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:36.011920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:36.011920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:36.011920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:36.011920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:36.011920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:36.011920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:36.011920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:36.011920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:36.011920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:36.011920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:36.011920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:36.011920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:36.011920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:36.011920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:36.011920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:36.012874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:36.012874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:36.012874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:36.012874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:36.012874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:36.012874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:36.012874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:36.012874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:36.012874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:36.012874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:36.012874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:18:36.012874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:36.013526 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ad6ee-512c-4ea2-a72b-1b2653c0189c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:36.013597 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 390/788] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:18:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:36.017865 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4485601-7058-4976-b8a6-e3e4dee57f10 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:36.035252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:36.123821 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4485601-7058-4976-b8a6-e3e4dee57f10 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:36.166833 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913bd0b-4b2e-4663-86d4-26814417db02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:36.167452 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913bd0b-4b2e-4663-86d4-26814417db02 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:36.167731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913bd0b-4b2e-4663-86d4-26814417db02 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:36.168005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913bd0b-4b2e-4663-86d4-26814417db02 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:36.168748 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913bd0b-4b2e-4663-86d4-26814417db02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:36.169109 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913bd0b-4b2e-4663-86d4-26814417db02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:36.171532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913bd0b-4b2e-4663-86d4-26814417db02 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:36.189007 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 391/789] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:18:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:36.194789 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913bd0b-4b2e-4663-86d4-26814417db02 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:36.255085 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ad6ee-512c-4ea2-a72b-1b2653c0189c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:36.287678 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:36.304003 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afae8ec-89d1-4f0c-9aa6-008d0779f935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:36.304694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afae8ec-89d1-4f0c-9aa6-008d0779f935 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:36.305020 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afae8ec-89d1-4f0c-9aa6-008d0779f935 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:36.305293 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afae8ec-89d1-4f0c-9aa6-008d0779f935 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:36.305983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afae8ec-89d1-4f0c-9aa6-008d0779f935 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:36.306283 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afae8ec-89d1-4f0c-9aa6-008d0779f935 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:36.307652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afae8ec-89d1-4f0c-9aa6-008d0779f935 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:36.337696 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 399/790] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:36.338077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:36.344293 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afae8ec-89d1-4f0c-9aa6-008d0779f935 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:36.373073 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913bd0b-4b2e-4663-86d4-26814417db02 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:36.407952 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848e575-a8d6-4461-80b8-f9131feb1611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:36.408662 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848e575-a8d6-4461-80b8-f9131feb1611 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:36.408940 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848e575-a8d6-4461-80b8-f9131feb1611 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:36.409212 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848e575-a8d6-4461-80b8-f9131feb1611 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:36.421144 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4848e575-a8d6-4461-80b8-f9131feb1611 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:36.422531 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4848e575-a8d6-4461-80b8-f9131feb1611 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:36.443230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-4848e575-a8d6-4461-80b8-f9131feb1611 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:36.443230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:36.443230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:36.443230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:36.443230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:36.443230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:36.443230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:36.443230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:36.443230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:36.443230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:36.443230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:36.443230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:36.443230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:36.443230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:36.443230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:36.444171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:36.444171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:36.444171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:36.444171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:36.444171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:36.444171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:36.444171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:36.444171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:36.444171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:36.444171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:36.444171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:36.444171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:36.444171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:36.444171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:36.444171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:36.444171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:36.444171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:36.445091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:36.445091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:36.445091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:36.445091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:36.447553 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 392/791] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:36.452968 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848e575-a8d6-4461-80b8-f9131feb1611 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:36.519208 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afae8ec-89d1-4f0c-9aa6-008d0779f935 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:36.556415 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec994989-5241-4bf2-941f-ab1085986baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:36.557962 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec994989-5241-4bf2-941f-ab1085986baf None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:36.558249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec994989-5241-4bf2-941f-ab1085986baf None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:36.558520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec994989-5241-4bf2-941f-ab1085986baf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:36.566682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec994989-5241-4bf2-941f-ab1085986baf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:36.568113 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec994989-5241-4bf2-941f-ab1085986baf None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:36.581878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-ec994989-5241-4bf2-941f-ab1085986baf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 12:18:36.581878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:36.581878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:36.581878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:36.581878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:36.581878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:36.581878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:36.581878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:36.581878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:36.581878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:36.581878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:36.581878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:36.581878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:36.581878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:36.581878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:36.581878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:36.582822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:36.582822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:36.582822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:36.582822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:36.582822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:36.582822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:36.582822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:36.582822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:36.582822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:36.582822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:36.582822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:36.582822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:36.582822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:36.582822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:36.582822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:36.582822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:36.582822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:36.582822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:36.583767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:36.583767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:36.583767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:36.583767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:36.583767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:36.583767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:36.583767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:36.583767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:36.583767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 12:18:36.583767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:36.583767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 400/792] 10.222.0.20 () {56 vars in 1117 bytes} [Sat Aug 6 12:18:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:36.593832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec994989-5241-4bf2-941f-ab1085986baf None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:36.668181 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:36.686760 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848e575-a8d6-4461-80b8-f9131feb1611 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:36.731547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:36.748067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb16b32-1c5c-47f0-bb62-73382a37e679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:36.748773 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb16b32-1c5c-47f0-bb62-73382a37e679 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:36.749047 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb16b32-1c5c-47f0-bb62-73382a37e679 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:36.749316 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb16b32-1c5c-47f0-bb62-73382a37e679 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:36.750010 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb16b32-1c5c-47f0-bb62-73382a37e679 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:36.750307 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb16b32-1c5c-47f0-bb62-73382a37e679 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:36.752679 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb16b32-1c5c-47f0-bb62-73382a37e679 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:36.771156 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 393/793] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:36.773040 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec994989-5241-4bf2-941f-ab1085986baf None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:36.775629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb16b32-1c5c-47f0-bb62-73382a37e679 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:36.810730 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05b0a1-9909-40b1-8fe0-cbecbec7aca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:36.811343 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05b0a1-9909-40b1-8fe0-cbecbec7aca9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:36.811620 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05b0a1-9909-40b1-8fe0-cbecbec7aca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:36.811888 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05b0a1-9909-40b1-8fe0-cbecbec7aca9 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:36.812761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff05b0a1-9909-40b1-8fe0-cbecbec7aca9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:36.813062 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff05b0a1-9909-40b1-8fe0-cbecbec7aca9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:36.814530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff05b0a1-9909-40b1-8fe0-cbecbec7aca9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:36.833122 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 401/794] 10.222.0.20 () {56 vars in 1149 bytes} [Sat Aug 6 12:18:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:36.838503 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05b0a1-9909-40b1-8fe0-cbecbec7aca9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:36.956272 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb16b32-1c5c-47f0-bb62-73382a37e679 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:36.972051 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:36.975933 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05b0a1-9909-40b1-8fe0-cbecbec7aca9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:36.998993 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f8144-4729-4398-9783-62e670a0f444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:36.999898 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f8144-4729-4398-9783-62e670a0f444 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:37.001000 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f8144-4729-4398-9783-62e670a0f444 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:37.001524 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f8144-4729-4398-9783-62e670a0f444 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:37.012070 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98f8144-4729-4398-9783-62e670a0f444 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:37.014291 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98f8144-4729-4398-9783-62e670a0f444 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:37.024491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247596e-ba25-45b6-8381-222dddc50d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:37.025135 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247596e-ba25-45b6-8381-222dddc50d4d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:37.025413 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247596e-ba25-45b6-8381-222dddc50d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:37.025686 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247596e-ba25-45b6-8381-222dddc50d4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:37.030060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-c98f8144-4729-4398-9783-62e670a0f444 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 12:18:37.030060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:37.030060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:37.030060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:37.030060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:37.030060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:37.030060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:37.030060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:37.030060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:37.030060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:37.030060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:37.030060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:37.030060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:37.030060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:37.030060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:37.030060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:37.031154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:37.031154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:37.031154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:37.031154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:37.031154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:37.031154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:37.031154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:37.031154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:37.031154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:37.031154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:37.031154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:37.031154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:37.031154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:37.031154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:37.031154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:37.031154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:37.031154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:37.031154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:37.032356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:37.032356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:37.032356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:37.032356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:37.032356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:37.032356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:37.032356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:37.032356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:37.032356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 12:18:37.032356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:37.035407 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 394/795] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:18:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:37.036790 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285d638-40aa-4a12-a32c-b7ac11bfdf0d None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:37.037255 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285d638-40aa-4a12-a32c-b7ac11bfdf0d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:37.037700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285d638-40aa-4a12-a32c-b7ac11bfdf0d None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:37.043277 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 395/796] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:37.043277 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a285d638-40aa-4a12-a32c-b7ac11bfdf0d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:37.044813 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8247596e-ba25-45b6-8381-222dddc50d4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:37.046253 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8247596e-ba25-45b6-8381-222dddc50d4d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:37.065769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-8247596e-ba25-45b6-8381-222dddc50d4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:37.065769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:37.065769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:37.065769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:37.065769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:37.065769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:37.065769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:37.065769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:37.065769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:37.065769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:37.065769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:37.065769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:37.065769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:37.065769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:37.065769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:37.066683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:37.066683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:37.066683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:37.066683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:37.066683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:37.066683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:37.066683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:37.066683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:37.066683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:37.066683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:37.066683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:37.066683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:37.066683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:37.066683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:37.066683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:37.066683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:37.066683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:37.067570 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:37.067570 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:37.067570 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:37.067570 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:37.069802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 402/797] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:37.071710 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b28225-c98f-4c91-8e6d-978520ed3853 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:37.072204 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b28225-c98f-4c91-8e6d-978520ed3853 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:37.072721 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b28225-c98f-4c91-8e6d-978520ed3853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:37.091298 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-56b28225-c98f-4c91-8e6d-978520ed3853 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:37.092777 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-56b28225-c98f-4c91-8e6d-978520ed3853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:37.096341 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:37.153441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a285d638-40aa-4a12-a32c-b7ac11bfdf0d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:37.172401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 403/798] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:18:37.173727 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b28225-c98f-4c91-8e6d-978520ed3853 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:37.198545 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c62003-d93e-4232-960f-289f26fd58e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:37.199162 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c62003-d93e-4232-960f-289f26fd58e1 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:37.199439 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c62003-d93e-4232-960f-289f26fd58e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:37.199711 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c62003-d93e-4232-960f-289f26fd58e1 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:37.200583 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c62003-d93e-4232-960f-289f26fd58e1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:37.201296 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c62003-d93e-4232-960f-289f26fd58e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:37.204176 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c62003-d93e-4232-960f-289f26fd58e1 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:37.226715 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 396/799] 10.222.0.20 () {56 vars in 1135 bytes} [Sat Aug 6 12:18:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:37.228032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c62003-d93e-4232-960f-289f26fd58e1 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:37.297215 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:37.326415 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b28225-c98f-4c91-8e6d-978520ed3853 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:37.351553 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c62003-d93e-4232-960f-289f26fd58e1 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:37.354173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:37.364082 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afa2eb-c289-472f-b1c4-4c84ea463442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:37.365035 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afa2eb-c289-472f-b1c4-4c84ea463442 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:37.365523 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afa2eb-c289-472f-b1c4-4c84ea463442 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:37.366360 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afa2eb-c289-472f-b1c4-4c84ea463442 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:37.368040 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40afa2eb-c289-472f-b1c4-4c84ea463442 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:37.368575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40afa2eb-c289-472f-b1c4-4c84ea463442 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:37.370256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40afa2eb-c289-472f-b1c4-4c84ea463442 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:37.383274 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 404/800] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:37.384435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afa2eb-c289-472f-b1c4-4c84ea463442 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:37.391277 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ceea4-61d4-4603-bc38-0b07767c2ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:37.391920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ceea4-61d4-4603-bc38-0b07767c2ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:37.392197 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ceea4-61d4-4603-bc38-0b07767c2ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:37.392499 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ceea4-61d4-4603-bc38-0b07767c2ef3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:37.393183 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ceea4-61d4-4603-bc38-0b07767c2ef3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:37.394548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ceea4-61d4-4603-bc38-0b07767c2ef3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:37.406532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-348ceea4-61d4-4603-bc38-0b07767c2ef3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 12:18:37.406532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:37.406532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:18:37.406532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:18:37.406532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:18:37.406532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:18:37.406532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:18:37.406532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:37.406532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 12:18:37.406532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:37.406532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:18:37.406532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:37.406532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:37.406532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:37.406532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:37.406532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:37.406532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:37.407536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:37.407536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:37.407536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:37.407536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:37.407536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:37.407536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:37.407536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:18:37.407536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:18:37.407536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:18:37.407536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:18:37.407536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:37.407536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:37.407536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:37.407536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:37.407536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:37.407536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:37.408394 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:37.408394 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:37.408394 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:37.408394 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:37.408394 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:37.408394 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:37.408394 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:37.408394 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:37.408394 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:37.408394 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:37.408394 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:18:37.408394 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:37.408394 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 06 12:18:37.408394 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:37.412429 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 397/801] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:37 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:37.413347 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ceea4-61d4-4603-bc38-0b07767c2ef3 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:37.500776 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afa2eb-c289-472f-b1c4-4c84ea463442 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:37.522808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ceea4-61d4-4603-bc38-0b07767c2ef3 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:37.538237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c174c6b-777a-4e99-adae-7fc6273f8b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:37.540149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c174c6b-777a-4e99-adae-7fc6273f8b4a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:37.540656 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c174c6b-777a-4e99-adae-7fc6273f8b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:37.541196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c174c6b-777a-4e99-adae-7fc6273f8b4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:37.551179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:37.553894 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c174c6b-777a-4e99-adae-7fc6273f8b4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:37.557157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c174c6b-777a-4e99-adae-7fc6273f8b4a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:37.560774 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee0aa6-8f89-4487-89ea-24c934a47b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:37.561407 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee0aa6-8f89-4487-89ea-24c934a47b80 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:37.561687 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee0aa6-8f89-4487-89ea-24c934a47b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:37.561955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee0aa6-8f89-4487-89ea-24c934a47b80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:37.569531 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ee0aa6-8f89-4487-89ea-24c934a47b80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:37.571080 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ee0aa6-8f89-4487-89ea-24c934a47b80 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:37.575360 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-3c174c6b-777a-4e99-adae-7fc6273f8b4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:37.575360 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:37.575360 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:37.575360 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:37.575360 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:37.575360 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:37.575360 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:37.575360 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:37.575360 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:37.575360 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:37.575360 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:37.575360 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:37.575360 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:37.575360 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:37.575360 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:37.576252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:37.576252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:37.576252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:37.576252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:37.576252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:37.576252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:37.576252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:37.576252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:37.576252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:37.576252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:37.576252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:37.576252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:37.576252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:37.576252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:37.576252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:37.576252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:37.576252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:37.577190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:37.577190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:37.577190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:37.577190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:37.579335 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 405/802] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:37.580405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c174c6b-777a-4e99-adae-7fc6273f8b4a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:37.587724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-00ee0aa6-8f89-4487-89ea-24c934a47b80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:37.587724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:37.587724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:37.587724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:37.587724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:37.587724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:37.587724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:37.587724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:37.587724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:37.587724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:37.587724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:37.587724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:37.587724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:37.587724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:37.587724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:37.588889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:37.588889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:37.588889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:37.588889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:37.588889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:37.588889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:37.588889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:37.588889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:37.588889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:37.588889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:37.588889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:37.588889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:37.588889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:37.588889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:37.588889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:37.588889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:37.588889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:37.589798 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:37.589798 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:37.589798 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:37.589798 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:37.589798 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:37.589798 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:37.589798 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:37.589798 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:37.589798 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 398/803] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:37.590295 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee0aa6-8f89-4487-89ea-24c934a47b80 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:37.621151 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:37.698094 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c174c6b-777a-4e99-adae-7fc6273f8b4a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:37.698457 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee0aa6-8f89-4487-89ea-24c934a47b80 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:37.734415 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78617925-9ba1-4078-9ba4-3941233fd797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:37.735163 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78617925-9ba1-4078-9ba4-3941233fd797 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:37.735540 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78617925-9ba1-4078-9ba4-3941233fd797 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:37.735964 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78617925-9ba1-4078-9ba4-3941233fd797 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:37.736249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe397a-3803-49a7-a0bc-801e627e83ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:37.736870 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78617925-9ba1-4078-9ba4-3941233fd797 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:37.737244 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe397a-3803-49a7-a0bc-801e627e83ed None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:37.737358 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78617925-9ba1-4078-9ba4-3941233fd797 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:37.737783 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe397a-3803-49a7-a0bc-801e627e83ed None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:37.738193 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe397a-3803-49a7-a0bc-801e627e83ed None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:37.738950 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78617925-9ba1-4078-9ba4-3941233fd797 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:37.739173 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe397a-3803-49a7-a0bc-801e627e83ed None admin] RBAC: Adding query filter params (name=network) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:37.739610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe397a-3803-49a7-a0bc-801e627e83ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:37.741464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe397a-3803-49a7-a0bc-801e627e83ed None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:37.746479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 406/804] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Aug 6 12:18:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:37.747441 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe397a-3803-49a7-a0bc-801e627e83ed None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:37.757517 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 399/805] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:37.758833 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78617925-9ba1-4078-9ba4-3941233fd797 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:37.809731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:37.857140 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:37.862270 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe397a-3803-49a7-a0bc-801e627e83ed None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:37.872693 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78617925-9ba1-4078-9ba4-3941233fd797 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:37.904465 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ca4ac-0074-498d-ac30-9260c3dc2a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:37.905080 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ca4ac-0074-498d-ac30-9260c3dc2a5e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:37.905357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ca4ac-0074-498d-ac30-9260c3dc2a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:37.905625 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ca4ac-0074-498d-ac30-9260c3dc2a5e None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:37.906367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ca4ac-0074-498d-ac30-9260c3dc2a5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:37.906482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c916ff9-3aa6-429c-9153-dbd7bf4a5528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:37.906669 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ca4ac-0074-498d-ac30-9260c3dc2a5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:37.907040 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c916ff9-3aa6-429c-9153-dbd7bf4a5528 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:37.907314 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c916ff9-3aa6-429c-9153-dbd7bf4a5528 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:37.907583 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c916ff9-3aa6-429c-9153-dbd7bf4a5528 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:37.908136 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ca4ac-0074-498d-ac30-9260c3dc2a5e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:37.908232 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c916ff9-3aa6-429c-9153-dbd7bf4a5528 None admin] RBAC: Adding query filter params (type=network) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:37.908566 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c916ff9-3aa6-429c-9153-dbd7bf4a5528 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:37.909901 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c916ff9-3aa6-429c-9153-dbd7bf4a5528 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:37.919789 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 400/806] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Aug 6 12:18:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:37.919789 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c916ff9-3aa6-429c-9153-dbd7bf4a5528 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:37.925288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 407/807] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:37.926507 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d096a2-55dd-4907-bc98-952ca4cb9c70 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:37.926781 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d096a2-55dd-4907-bc98-952ca4cb9c70 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:37.927052 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d096a2-55dd-4907-bc98-952ca4cb9c70 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:37.928585 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 408/808] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:37.929712 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d096a2-55dd-4907-bc98-952ca4cb9c70 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:38.027416 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c916ff9-3aa6-429c-9153-dbd7bf4a5528 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:38.045108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d096a2-55dd-4907-bc98-952ca4cb9c70 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:38.062571 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5b28b-173d-4943-bc9c-30feb6c78ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:38.063204 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5b28b-173d-4943-bc9c-30feb6c78ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:38.063484 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5b28b-173d-4943-bc9c-30feb6c78ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:38.063767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5b28b-173d-4943-bc9c-30feb6c78ee4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:38.064769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:38.072230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5b28b-173d-4943-bc9c-30feb6c78ee4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:38.073647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5b28b-173d-4943-bc9c-30feb6c78ee4 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:38.083339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4213a1-726f-4c60-9533-255f26d137f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:38.083950 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4213a1-726f-4c60-9533-255f26d137f8 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:38.084228 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4213a1-726f-4c60-9533-255f26d137f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:38.084539 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4213a1-726f-4c60-9533-255f26d137f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:38.085236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4213a1-726f-4c60-9533-255f26d137f8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:38.085630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-cda5b28b-173d-4943-bc9c-30feb6c78ee4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:38.085630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:38.085630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:38.085630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:38.085630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:38.085630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:38.085630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:38.085630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:38.085630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:38.085630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:38.085630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:38.085630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:38.085630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:38.085630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:38.085630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:38.086503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:38.086503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:38.086503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:38.086503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:38.086503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:38.086503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:38.086503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:38.086503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:38.086503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:38.086503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:38.086503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:38.086503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:38.086503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:38.086503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:38.086503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:38.086503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:38.086503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:38.087391 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:38.087391 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:38.087391 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:38.087391 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:38.087391 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:38.087391 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:38.087391 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:38.087391 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:38.087815 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4213a1-726f-4c60-9533-255f26d137f8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:38.087883 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 401/809] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:38.088737 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5b28b-173d-4943-bc9c-30feb6c78ee4 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:38.109204 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:38.109624 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 409/810] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:18:37 2022] POST /identity/v3/endpoints => generated 337 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:18:38.111367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd968d-1bae-4abf-869f-37612c90ad31 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:38.111703 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd968d-1bae-4abf-869f-37612c90ad31 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:38.111970 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd968d-1bae-4abf-869f-37612c90ad31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:38.136829 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-fccd968d-1bae-4abf-869f-37612c90ad31 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:38.138058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fccd968d-1bae-4abf-869f-37612c90ad31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:38.207937 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5b28b-173d-4943-bc9c-30feb6c78ee4 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:38.228094 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 410/811] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:18:38.229517 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6207a5-a39c-4af6-8074-a2c6aaeae203 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:38.229807 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6207a5-a39c-4af6-8074-a2c6aaeae203 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:38.230084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6207a5-a39c-4af6-8074-a2c6aaeae203 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:38.231529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 411/812] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:38.233192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6207a5-a39c-4af6-8074-a2c6aaeae203 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:38.261540 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ca93e-bb6c-4343-af91-aa3ba04a01cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:38.262232 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ca93e-bb6c-4343-af91-aa3ba04a01cf None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:38.262514 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ca93e-bb6c-4343-af91-aa3ba04a01cf None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:38.262791 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ca93e-bb6c-4343-af91-aa3ba04a01cf None admin] PATH_INFO: `/v3/projects/13fba33c46cf40bbb15a08c690b4c98c/users/c810b52785b94c0cab4fd9f0d6e65825/roles/785cca921bad4984a86e45ce6c5601c2` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:38.279417 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ca93e-bb6c-4343-af91-aa3ba04a01cf None admin] RBAC: Authorizing `identity:create_grant(project_id=13fba33c46cf40bbb15a08c690b4c98c, user_id=c810b52785b94c0cab4fd9f0d6e65825, role_id=785cca921bad4984a86e45ce6c5601c2)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:38.280565 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ca93e-bb6c-4343-af91-aa3ba04a01cf None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:38.295044 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 402/813] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:18:38 2022] PUT /identity/v3/projects/13fba33c46cf40bbb15a08c690b4c98c/users/c810b52785b94c0cab4fd9f0d6e65825/roles/785cca921bad4984a86e45ce6c5601c2 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:18:38.295718 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b51171-1e84-4ee3-9024-e5324e31929f None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:38.296004 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b51171-1e84-4ee3-9024-e5324e31929f None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:38.296278 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b51171-1e84-4ee3-9024-e5324e31929f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:38.315257 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-68b51171-1e84-4ee3-9024-e5324e31929f None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:38.317390 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-68b51171-1e84-4ee3-9024-e5324e31929f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:38.373023 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:38.375537 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6207a5-a39c-4af6-8074-a2c6aaeae203 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:38.394665 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:38.408502 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 403/814] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:18:38.408502 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b51171-1e84-4ee3-9024-e5324e31929f None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:38.427001 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bdd69e-b0c9-4592-ad57-13989b86ebb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:38.427956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdd69e-b0c9-4592-ad57-13989b86ebb4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:38.428422 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdd69e-b0c9-4592-ad57-13989b86ebb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:38.428848 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdd69e-b0c9-4592-ad57-13989b86ebb4 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:38.429783 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bdd69e-b0c9-4592-ad57-13989b86ebb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:38.430383 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bdd69e-b0c9-4592-ad57-13989b86ebb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:38.434183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bdd69e-b0c9-4592-ad57-13989b86ebb4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:38.456401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 412/815] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:38.456889 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bdd69e-b0c9-4592-ad57-13989b86ebb4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:38.529685 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b51171-1e84-4ee3-9024-e5324e31929f None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:38.567943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da22709-f7ee-4c7d-bc1c-30694f8f33d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:38.568653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da22709-f7ee-4c7d-bc1c-30694f8f33d0 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:38.568935 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da22709-f7ee-4c7d-bc1c-30694f8f33d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:38.569202 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da22709-f7ee-4c7d-bc1c-30694f8f33d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:38.573067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bdd69e-b0c9-4592-ad57-13989b86ebb4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:38.580399 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da22709-f7ee-4c7d-bc1c-30694f8f33d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:38.585312 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-3da22709-f7ee-4c7d-bc1c-30694f8f33d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:18:38.585312 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:38.585312 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:38.585312 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:38.585312 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:38.585312 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:38.585312 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:38.585312 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:38.585312 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:38.585312 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:38.585312 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:38.585312 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:38.585312 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:38.585312 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:38.585312 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:38.585312 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:38.586825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:38.586825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:38.586825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:38.586825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:38.586825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:38.586825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:38.586825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:38.586825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:38.586825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:38.586825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:38.586825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:38.586825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:38.586825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:38.586825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:38.586825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:38.586825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:38.587686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:38.587686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:38.587686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:38.587686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:38.587686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:38.587686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:38.587686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:38.587686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:38.587686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:38.587686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:38.587686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:18:38.587686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:38.587686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 404/816] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:18:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:38.588434 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da22709-f7ee-4c7d-bc1c-30694f8f33d0 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:38.614857 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8116e-f8d9-47bb-81f2-c6b7b4bdfc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:38.615497 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8116e-f8d9-47bb-81f2-c6b7b4bdfc63 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:38.615776 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8116e-f8d9-47bb-81f2-c6b7b4bdfc63 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:38.616051 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8116e-f8d9-47bb-81f2-c6b7b4bdfc63 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:38.616957 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8116e-f8d9-47bb-81f2-c6b7b4bdfc63 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:38.619541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8116e-f8d9-47bb-81f2-c6b7b4bdfc63 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:38.629732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-30a8116e-f8d9-47bb-81f2-c6b7b4bdfc63 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 12:18:38.629732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:38.629732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:18:38.629732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:18:38.629732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:18:38.629732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:18:38.629732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:18:38.629732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:38.629732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 12:18:38.629732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:38.629732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:18:38.629732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:38.629732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:38.629732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:38.629732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:38.629732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:38.630709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:38.630709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:38.630709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:38.630709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:38.630709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:38.630709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:38.630709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:38.630709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:18:38.630709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:18:38.630709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:18:38.630709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:18:38.630709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:38.630709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:38.630709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:38.630709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:38.630709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:38.631562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:38.631562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:38.631562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:38.631562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:38.631562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:38.631562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:38.631562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:38.631562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:38.631562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:38.631562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:38.631562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:38.631562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:18:38.631562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:38.631562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 06 12:18:38.631562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:38.635858 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 413/817] 10.222.0.20 () {56 vars in 1129 bytes} [Sat Aug 6 12:18:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:38.636751 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8116e-f8d9-47bb-81f2-c6b7b4bdfc63 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:38.649160 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:38.663144 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:38.701853 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da22709-f7ee-4c7d-bc1c-30694f8f33d0 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:38.737906 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7c75d-14a1-4b96-b744-aac4b98516ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:38.738572 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7c75d-14a1-4b96-b744-aac4b98516ee None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:38.738854 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7c75d-14a1-4b96-b744-aac4b98516ee None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:38.739130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7c75d-14a1-4b96-b744-aac4b98516ee None admin] PATH_INFO: `/v3/projects/13fba33c46cf40bbb15a08c690b4c98c/users/2efc792a2655439f8b8070010059b6bb/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:38.741445 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7c75d-14a1-4b96-b744-aac4b98516ee None admin] RBAC: Authorizing `identity:create_grant(project_id=13fba33c46cf40bbb15a08c690b4c98c, user_id=2efc792a2655439f8b8070010059b6bb, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:38.742936 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7c75d-14a1-4b96-b744-aac4b98516ee None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:38.745228 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8116e-f8d9-47bb-81f2-c6b7b4bdfc63 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:38.760356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 405/818] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:18:38 2022] PUT /identity/v3/projects/13fba33c46cf40bbb15a08c690b4c98c/users/2efc792a2655439f8b8070010059b6bb/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:18:38.760356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7c75d-14a1-4b96-b744-aac4b98516ee None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:38.811671 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01764fe5-cd54-430b-a22d-e54f755d65f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:38.813204 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01764fe5-cd54-430b-a22d-e54f755d65f0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:38.813890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01764fe5-cd54-430b-a22d-e54f755d65f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:38.814334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01764fe5-cd54-430b-a22d-e54f755d65f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:38.815331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01764fe5-cd54-430b-a22d-e54f755d65f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:38.815792 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01764fe5-cd54-430b-a22d-e54f755d65f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:38.817390 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01764fe5-cd54-430b-a22d-e54f755d65f0 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:38.829444 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 414/819] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:18:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:38.834703 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01764fe5-cd54-430b-a22d-e54f755d65f0 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:38.899710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7c75d-14a1-4b96-b744-aac4b98516ee None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:38.915387 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:38.948105 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a72504-1c4a-40f4-baf9-ce7effa0fce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:38.948861 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a72504-1c4a-40f4-baf9-ce7effa0fce4 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:38.949189 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a72504-1c4a-40f4-baf9-ce7effa0fce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:38.949494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a72504-1c4a-40f4-baf9-ce7effa0fce4 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:38.950210 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a72504-1c4a-40f4-baf9-ce7effa0fce4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:38.950515 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a72504-1c4a-40f4-baf9-ce7effa0fce4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:38.952087 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a72504-1c4a-40f4-baf9-ce7effa0fce4 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:38.957865 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 406/820] 10.222.0.20 () {56 vars in 1124 bytes} [Sat Aug 6 12:18:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:38.963525 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a72504-1c4a-40f4-baf9-ce7effa0fce4 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:38.965541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:38.985243 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01764fe5-cd54-430b-a22d-e54f755d65f0 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:39.026446 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d858d-d653-44cf-b8bd-491547a50ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:39.027306 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d858d-d653-44cf-b8bd-491547a50ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:39.027418 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d858d-d653-44cf-b8bd-491547a50ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:39.027687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d858d-d653-44cf-b8bd-491547a50ee1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:39.039981 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d858d-d653-44cf-b8bd-491547a50ee1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:39.041514 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d858d-d653-44cf-b8bd-491547a50ee1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:39.056171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-cd3d858d-d653-44cf-b8bd-491547a50ee1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:39.056171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:39.056171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:39.056171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:39.056171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:39.056171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:39.056171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:39.056171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:39.056171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:39.056171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:39.056171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:39.056171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:39.056171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:39.056171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:39.056171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:39.057128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:39.057128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:39.057128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:39.057128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:39.057128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:39.057128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:39.057128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:39.057128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:39.057128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:39.057128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:39.057128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:39.057128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:39.057128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:39.057128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:39.057128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:39.057128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:39.057128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:39.058023 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:39.058023 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:39.058023 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:39.058023 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:39.058023 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 415/821] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:39.066893 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d858d-d653-44cf-b8bd-491547a50ee1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:39.097217 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a72504-1c4a-40f4-baf9-ce7effa0fce4 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:39.143934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d5602c-bce4-48ce-91fd-8ab9f487f592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:39.145098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d5602c-bce4-48ce-91fd-8ab9f487f592 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:39.145575 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d5602c-bce4-48ce-91fd-8ab9f487f592 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:39.146812 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d5602c-bce4-48ce-91fd-8ab9f487f592 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:39.147675 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d5602c-bce4-48ce-91fd-8ab9f487f592 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:39.148146 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d5602c-bce4-48ce-91fd-8ab9f487f592 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:39.150325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d5602c-bce4-48ce-91fd-8ab9f487f592 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:39.164011 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 407/822] 10.222.0.20 () {56 vars in 1124 bytes} [Sat Aug 6 12:18:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:39.164757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d5602c-bce4-48ce-91fd-8ab9f487f592 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:39.193130 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d858d-d653-44cf-b8bd-491547a50ee1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:39.213505 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:39.243227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef8b43-c107-474d-b89e-cf9f295de18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:39.245059 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef8b43-c107-474d-b89e-cf9f295de18e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:39.245552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef8b43-c107-474d-b89e-cf9f295de18e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:39.246002 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef8b43-c107-474d-b89e-cf9f295de18e None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:39.246866 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef8b43-c107-474d-b89e-cf9f295de18e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:39.247350 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef8b43-c107-474d-b89e-cf9f295de18e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:39.249473 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef8b43-c107-474d-b89e-cf9f295de18e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:39.253017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:39.267847 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 416/823] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:39.276011 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef8b43-c107-474d-b89e-cf9f295de18e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:39.281891 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d5602c-bce4-48ce-91fd-8ab9f487f592 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:39.320050 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854a1b1-978e-4c8a-bff4-861abf783728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:39.321367 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854a1b1-978e-4c8a-bff4-861abf783728 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:39.321798 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854a1b1-978e-4c8a-bff4-861abf783728 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:39.322214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854a1b1-978e-4c8a-bff4-861abf783728 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:39.323227 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854a1b1-978e-4c8a-bff4-861abf783728 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:39.324756 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854a1b1-978e-4c8a-bff4-861abf783728 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:39.350840 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 408/824] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:18:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 12:18:39.399246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef8b43-c107-474d-b89e-cf9f295de18e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:39.448405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de4f61-a4a0-4455-b403-17ef2b0c9c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:39.449240 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de4f61-a4a0-4455-b403-17ef2b0c9c56 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:39.449623 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de4f61-a4a0-4455-b403-17ef2b0c9c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:39.450016 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de4f61-a4a0-4455-b403-17ef2b0c9c56 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:39.459317 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de4f61-a4a0-4455-b403-17ef2b0c9c56 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:39.461857 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de4f61-a4a0-4455-b403-17ef2b0c9c56 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:39.475694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-05de4f61-a4a0-4455-b403-17ef2b0c9c56 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 12:18:39.475694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:39.475694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:39.475694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:39.475694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:39.475694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:39.475694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:39.475694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:39.475694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:39.475694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:39.475694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:39.475694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:39.475694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:39.475694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:39.475694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:39.475694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:39.477043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:39.477043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:39.477043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:39.477043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:39.477043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:39.477043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:39.477043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:39.477043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:39.477043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:39.477043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:39.477043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:39.477043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:39.477043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:39.477043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:39.477043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:39.477043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:39.477043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:39.477043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:39.478185 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:39.478185 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:39.478185 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:39.478185 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:39.478185 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:39.478185 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:39.478185 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:39.478185 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:39.478185 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 12:18:39.478185 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:39.483770 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 417/825] 10.222.0.20 () {56 vars in 1093 bytes} [Sat Aug 6 12:18:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:39.486192 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854a1b1-978e-4c8a-bff4-861abf783728 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:39.585555 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:39.593013 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854a1b1-978e-4c8a-bff4-861abf783728 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:39.629478 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a8946-b846-46c1-8edd-257df7c6098c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:39.630094 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a8946-b846-46c1-8edd-257df7c6098c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:39.630373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a8946-b846-46c1-8edd-257df7c6098c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:39.630651 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a8946-b846-46c1-8edd-257df7c6098c None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:39.631442 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a8946-b846-46c1-8edd-257df7c6098c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:39.631747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a8946-b846-46c1-8edd-257df7c6098c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:39.633246 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a8946-b846-46c1-8edd-257df7c6098c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:39.651769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 409/826] 10.222.0.20 () {56 vars in 1133 bytes} [Sat Aug 6 12:18:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:39.657454 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de4f61-a4a0-4455-b403-17ef2b0c9c56 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:39.713737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:39.765207 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de4f61-a4a0-4455-b403-17ef2b0c9c56 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:39.800832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13d612-1011-47ec-a19e-9b8a87bd3aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:39.801454 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13d612-1011-47ec-a19e-9b8a87bd3aed None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:39.801779 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13d612-1011-47ec-a19e-9b8a87bd3aed None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:39.802060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13d612-1011-47ec-a19e-9b8a87bd3aed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:39.874322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e13d612-1011-47ec-a19e-9b8a87bd3aed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:39.875700 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e13d612-1011-47ec-a19e-9b8a87bd3aed None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:39.930376 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:39.984022 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 418/827] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:18:39 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 12:18:39.989813 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a8946-b846-46c1-8edd-257df7c6098c None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:40.098791 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a8946-b846-46c1-8edd-257df7c6098c None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:40.134326 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab01381-0d99-4cf4-a59a-7ed9cf00c945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:40.135251 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab01381-0d99-4cf4-a59a-7ed9cf00c945 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:40.135755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab01381-0d99-4cf4-a59a-7ed9cf00c945 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:40.136222 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab01381-0d99-4cf4-a59a-7ed9cf00c945 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:40.146743 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab01381-0d99-4cf4-a59a-7ed9cf00c945 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:40.148416 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab01381-0d99-4cf4-a59a-7ed9cf00c945 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:40.163972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-4ab01381-0d99-4cf4-a59a-7ed9cf00c945 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:40.163972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:40.163972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:40.163972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:40.163972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:40.163972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:40.163972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:40.163972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:40.163972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:40.163972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:40.163972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:40.163972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:40.163972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:40.163972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:40.163972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:40.164966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:40.164966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:40.164966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:40.164966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:40.164966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:40.164966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:40.164966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:40.164966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:40.164966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:40.164966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:40.164966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:40.164966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:40.164966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:40.164966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:40.164966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:40.164966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:40.164966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:40.165875 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:40.165875 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:40.165875 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:40.165875 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:40.167750 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 410/828] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:40.173466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13d612-1011-47ec-a19e-9b8a87bd3aed None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:40.276826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13d612-1011-47ec-a19e-9b8a87bd3aed None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:40.312056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6c23a-ee9a-4b20-862f-8dee12c49152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:40.312931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6c23a-ee9a-4b20-862f-8dee12c49152 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:40.313391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6c23a-ee9a-4b20-862f-8dee12c49152 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:40.313803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6c23a-ee9a-4b20-862f-8dee12c49152 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:40.314657 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6c23a-ee9a-4b20-862f-8dee12c49152 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:40.315102 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6c23a-ee9a-4b20-862f-8dee12c49152 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:40.316614 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6c23a-ee9a-4b20-862f-8dee12c49152 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:40.329382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 419/829] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:40.337361 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab01381-0d99-4cf4-a59a-7ed9cf00c945 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:40.349231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:40.414466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232a4d0-e14f-45ea-aa3e-95c440064af1 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:40.414711 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232a4d0-e14f-45ea-aa3e-95c440064af1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:40.415226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232a4d0-e14f-45ea-aa3e-95c440064af1 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:40.416825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 420/830] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:40.422497 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400ac8b-2f57-4b91-a193-d530345d299c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:40.422829 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400ac8b-2f57-4b91-a193-d530345d299c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:40.423184 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400ac8b-2f57-4b91-a193-d530345d299c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:40.445390 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1400ac8b-2f57-4b91-a193-d530345d299c None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:40.447753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1400ac8b-2f57-4b91-a193-d530345d299c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:40.455485 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab01381-0d99-4cf4-a59a-7ed9cf00c945 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:40.456848 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:40.495283 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389e3ad-1615-4e52-b792-e094462119f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:40.496291 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389e3ad-1615-4e52-b792-e094462119f4 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:40.496800 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389e3ad-1615-4e52-b792-e094462119f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:40.497281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389e3ad-1615-4e52-b792-e094462119f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:40.504933 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4389e3ad-1615-4e52-b792-e094462119f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:40.506859 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4389e3ad-1615-4e52-b792-e094462119f4 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:40.520776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-4389e3ad-1615-4e52-b792-e094462119f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:40.520776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:40.520776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:40.520776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:40.520776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:40.520776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:40.520776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:40.520776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:40.520776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:40.520776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:40.520776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:40.520776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:40.520776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:40.520776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:40.520776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:40.521699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:40.521699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:40.521699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:40.521699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:40.521699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:40.521699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:40.521699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:40.521699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:40.521699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:40.521699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:40.521699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:40.521699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:40.521699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:40.521699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:40.521699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:40.521699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:40.521699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:40.523016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:40.523016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:40.523016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:40.523016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:40.523016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:40.523016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:40.523016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:40.523016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:40.526988 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 411/831] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:40.528056 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389e3ad-1615-4e52-b792-e094462119f4 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:40.532276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 421/832] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 12:18:40.552160 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400ac8b-2f57-4b91-a193-d530345d299c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:40.614956 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:40.654645 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389e3ad-1615-4e52-b792-e094462119f4 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:40.683675 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400ac8b-2f57-4b91-a193-d530345d299c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:40.691692 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b8d1a-376e-472b-bf3f-349b670c4f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:40.692513 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b8d1a-376e-472b-bf3f-349b670c4f28 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:40.692889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b8d1a-376e-472b-bf3f-349b670c4f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:40.693263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b8d1a-376e-472b-bf3f-349b670c4f28 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:40.694106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b8d1a-376e-472b-bf3f-349b670c4f28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:40.694510 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b8d1a-376e-472b-bf3f-349b670c4f28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:40.696832 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b8d1a-376e-472b-bf3f-349b670c4f28 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:40.712809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 412/833] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:40.715625 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b8d1a-376e-472b-bf3f-349b670c4f28 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:40.726143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b2ea1-174a-4134-b182-6b144a722433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:40.727914 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b2ea1-174a-4134-b182-6b144a722433 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:40.728393 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b2ea1-174a-4134-b182-6b144a722433 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:40.729193 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b2ea1-174a-4134-b182-6b144a722433 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:40.745593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b2ea1-174a-4134-b182-6b144a722433 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:40.753348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-670b2ea1-174a-4134-b182-6b144a722433 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:40.753348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:40.753348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:40.753348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:40.753348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:40.753348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:40.753348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:40.753348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:40.753348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:40.753348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:40.753348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:40.753348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:40.753348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:40.753348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:40.753348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:40.753348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:40.754449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:40.754449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:40.754449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:40.754449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:40.754449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:40.754449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:40.754449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:40.754449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:40.754449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:40.754449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:40.754449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:40.754449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:40.754449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:40.754449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:40.754449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:40.754449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:40.755472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:40.755472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:40.755472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:40.755472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:40.755472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:40.755472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:40.755472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:40.755472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:40.755472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:40.755472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:40.755472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:18:40.755472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:40.758649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 422/834] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Aug 6 12:18:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:40.764762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b2ea1-174a-4134-b182-6b144a722433 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:40.768154 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:40.852535 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b8d1a-376e-472b-bf3f-349b670c4f28 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:40.874229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b2ea1-174a-4134-b182-6b144a722433 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:40.883074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:40.892080 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0aac0-b5c5-4029-b4ef-e5bb3e5b4e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:40.892745 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0aac0-b5c5-4029-b4ef-e5bb3e5b4e2d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:40.893024 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0aac0-b5c5-4029-b4ef-e5bb3e5b4e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:40.893288 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0aac0-b5c5-4029-b4ef-e5bb3e5b4e2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:40.894069 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0aac0-b5c5-4029-b4ef-e5bb3e5b4e2d None admin] RBAC: Adding query filter params (role.id=144c8425688b409384e1e8ed8279d88e, scope.project.id=13fba33c46cf40bbb15a08c690b4c98c, user.id=524b728bf63c4ca88e0f25e531cf5056) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:40.894382 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0aac0-b5c5-4029-b4ef-e5bb3e5b4e2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:40.895809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0aac0-b5c5-4029-b4ef-e5bb3e5b4e2d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:40.902625 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 413/835] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:18:40 2022] GET /identity/v3/role_assignments?user.id=524b728bf63c4ca88e0f25e531cf5056&scope.project.id=13fba33c46cf40bbb15a08c690b4c98c&role.id=144c8425688b409384e1e8ed8279d88e => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:40.904021 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43085e-e26d-4080-b03c-7465c68291a3 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:40.904326 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43085e-e26d-4080-b03c-7465c68291a3 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:40.904676 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43085e-e26d-4080-b03c-7465c68291a3 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:40.906023 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 414/836] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:40.914392 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acbd30-eec4-4070-ae0a-d9e772372e75 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:40.915468 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acbd30-eec4-4070-ae0a-d9e772372e75 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:40.916074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acbd30-eec4-4070-ae0a-d9e772372e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:40.924024 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88aa401-b97a-469e-9ae8-c9a91d34121c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:40.924918 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88aa401-b97a-469e-9ae8-c9a91d34121c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:40.926191 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88aa401-b97a-469e-9ae8-c9a91d34121c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:40.926191 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88aa401-b97a-469e-9ae8-c9a91d34121c None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:40.926604 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88aa401-b97a-469e-9ae8-c9a91d34121c None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:40.927011 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88aa401-b97a-469e-9ae8-c9a91d34121c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:40.928560 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88aa401-b97a-469e-9ae8-c9a91d34121c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:40.939276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 423/837] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Aug 6 12:18:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:40.940768 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-33acbd30-eec4-4070-ae0a-d9e772372e75 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:40.942097 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-33acbd30-eec4-4070-ae0a-d9e772372e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:40.944479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88aa401-b97a-469e-9ae8-c9a91d34121c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:41.033610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:41.044344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 415/838] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 12:18:41.066938 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33acbd30-eec4-4070-ae0a-d9e772372e75 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:41.102449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88aa401-b97a-469e-9ae8-c9a91d34121c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:41.149195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61488da-9c63-4f9d-86c4-86f63d13ebe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:41.149973 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61488da-9c63-4f9d-86c4-86f63d13ebe8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:41.150250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61488da-9c63-4f9d-86c4-86f63d13ebe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:41.150515 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61488da-9c63-4f9d-86c4-86f63d13ebe8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:41.163210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61488da-9c63-4f9d-86c4-86f63d13ebe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:41.165338 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61488da-9c63-4f9d-86c4-86f63d13ebe8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:41.182305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-e61488da-9c63-4f9d-86c4-86f63d13ebe8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:41.182305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:41.182305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:41.182305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:41.182305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:41.182305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:41.182305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:41.182305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:41.182305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:41.182305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:41.182305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:41.182305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:41.182305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:41.182305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:41.182305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:41.183251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:41.183251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:41.183251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:41.183251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:41.183251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:41.183251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:41.183251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:41.183251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:41.183251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:41.183251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:41.183251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:41.183251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:41.183251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:41.183251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:41.183251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:41.183251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:41.183251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:41.184146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:41.184146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:41.184146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:41.184146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:41.184146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 424/839] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:41.189389 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61488da-9c63-4f9d-86c4-86f63d13ebe8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:41.206123 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33acbd30-eec4-4070-ae0a-d9e772372e75 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:41.229059 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:41.243844 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c77f3e-571a-4205-aadb-6ddbcee54237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:41.244511 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c77f3e-571a-4205-aadb-6ddbcee54237 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:41.244788 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c77f3e-571a-4205-aadb-6ddbcee54237 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:41.245058 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c77f3e-571a-4205-aadb-6ddbcee54237 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:41.256762 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c77f3e-571a-4205-aadb-6ddbcee54237 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:41.262411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-c4c77f3e-571a-4205-aadb-6ddbcee54237 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:18:41.262411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:41.262411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:41.262411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:41.262411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:41.262411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:41.262411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:41.262411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:41.262411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:41.262411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:41.262411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:41.262411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:41.262411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:41.262411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:41.262411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:41.262411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:41.263345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:41.263345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:41.263345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:41.263345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:41.263345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:41.263345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:41.263345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:41.263345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:41.263345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:41.263345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:41.263345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:41.263345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:41.263345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:41.263345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:41.263345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:41.263345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:41.264175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:41.264175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:41.264175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:41.264175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:41.264175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:41.264175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:41.264175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:41.264175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:41.264175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:41.264175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:41.264175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:18:41.264175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:41.264175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 416/840] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:18:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:41.273443 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c77f3e-571a-4205-aadb-6ddbcee54237 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:41.321112 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61488da-9c63-4f9d-86c4-86f63d13ebe8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:41.338165 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:41.357808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117e68d-197e-441f-821b-2dd9639553f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:41.358264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117e68d-197e-441f-821b-2dd9639553f8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:41.358548 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117e68d-197e-441f-821b-2dd9639553f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:41.358823 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117e68d-197e-441f-821b-2dd9639553f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:41.359520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1117e68d-197e-441f-821b-2dd9639553f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:41.359820 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1117e68d-197e-441f-821b-2dd9639553f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:41.361311 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1117e68d-197e-441f-821b-2dd9639553f8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:41.378991 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 425/841] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:41.380058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117e68d-197e-441f-821b-2dd9639553f8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:41.389852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c77f3e-571a-4205-aadb-6ddbcee54237 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:41.430371 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07a254-2236-4f3a-b94b-3d4374f7606a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:41.431332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07a254-2236-4f3a-b94b-3d4374f7606a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:41.431984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07a254-2236-4f3a-b94b-3d4374f7606a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:41.432500 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07a254-2236-4f3a-b94b-3d4374f7606a None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:41.433465 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07a254-2236-4f3a-b94b-3d4374f7606a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:41.434059 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07a254-2236-4f3a-b94b-3d4374f7606a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:41.435787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07a254-2236-4f3a-b94b-3d4374f7606a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:41.455978 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 417/842] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:18:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:41.457504 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07a254-2236-4f3a-b94b-3d4374f7606a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:41.505996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117e68d-197e-441f-821b-2dd9639553f8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:41.513925 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:41.543171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758f45d-87e7-47fd-be70-28e6f0c21e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:41.544199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758f45d-87e7-47fd-be70-28e6f0c21e23 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:41.544199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758f45d-87e7-47fd-be70-28e6f0c21e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:41.544412 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758f45d-87e7-47fd-be70-28e6f0c21e23 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:41.551820 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758f45d-87e7-47fd-be70-28e6f0c21e23 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:41.554112 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758f45d-87e7-47fd-be70-28e6f0c21e23 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:41.566946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-7758f45d-87e7-47fd-be70-28e6f0c21e23 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 12:18:41.566946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:41.566946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:41.566946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:41.566946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:41.566946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:41.566946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:41.566946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:41.566946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:41.566946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:41.566946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:41.566946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:41.566946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:41.566946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:41.566946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:41.566946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:41.567883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:41.567883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:41.567883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:41.567883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:41.567883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:41.567883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:41.567883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:41.567883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:41.567883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:41.567883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:41.567883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:41.567883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:41.567883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:41.567883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:41.567883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:41.567883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:41.567883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:41.567883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:41.569586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:41.569586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:41.569586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:41.569586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:41.569586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:41.569586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:41.569586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:41.569586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:41.569586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 12:18:41.569586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:41.569586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 426/843] 10.222.0.20 () {56 vars in 1117 bytes} [Sat Aug 6 12:18:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:41.569586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5a2fd-47ce-408a-8e2f-a87f5c44f582 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:41.570229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5a2fd-47ce-408a-8e2f-a87f5c44f582 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:41.570229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5a2fd-47ce-408a-8e2f-a87f5c44f582 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:41.571981 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 427/844] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:41.576793 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5a2fd-47ce-408a-8e2f-a87f5c44f582 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:41.582202 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07a254-2236-4f3a-b94b-3d4374f7606a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:41.616755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deddda4e-af5a-4ba0-a567-50297800ff1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:41.617822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:41.617987 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deddda4e-af5a-4ba0-a567-50297800ff1c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:41.618167 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deddda4e-af5a-4ba0-a567-50297800ff1c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:41.618438 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deddda4e-af5a-4ba0-a567-50297800ff1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:41.630325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deddda4e-af5a-4ba0-a567-50297800ff1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:41.631774 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deddda4e-af5a-4ba0-a567-50297800ff1c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:41.646270 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-deddda4e-af5a-4ba0-a567-50297800ff1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:41.646270 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:41.646270 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:41.646270 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:41.646270 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:41.646270 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:41.646270 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:41.646270 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:41.646270 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:41.646270 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:41.646270 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:41.646270 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:41.646270 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:41.646270 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:41.646270 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:41.647180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:41.647180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:41.647180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:41.647180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:41.647180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:41.647180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:41.647180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:41.647180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:41.647180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:41.647180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:41.647180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:41.647180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:41.647180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:41.647180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:41.647180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:41.647180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:41.647180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:41.648077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:41.648077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:41.648077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:41.648077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:41.648077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 418/845] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:41.649426 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348f487-fd5b-4ee6-8ac8-6e9e07ffc85d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:41.649426 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348f487-fd5b-4ee6-8ac8-6e9e07ffc85d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:41.649641 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348f487-fd5b-4ee6-8ac8-6e9e07ffc85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:41.667870 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-7348f487-fd5b-4ee6-8ac8-6e9e07ffc85d None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:41.669095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7348f487-fd5b-4ee6-8ac8-6e9e07ffc85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:41.697306 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5a2fd-47ce-408a-8e2f-a87f5c44f582 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:41.735323 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14932bb2-5317-4b82-ae60-95a6c56a7737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:41.735954 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14932bb2-5317-4b82-ae60-95a6c56a7737 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:41.736161 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14932bb2-5317-4b82-ae60-95a6c56a7737 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:41.736443 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14932bb2-5317-4b82-ae60-95a6c56a7737 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:41.737311 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14932bb2-5317-4b82-ae60-95a6c56a7737 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:41.737554 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14932bb2-5317-4b82-ae60-95a6c56a7737 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:41.738974 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14932bb2-5317-4b82-ae60-95a6c56a7737 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:41.742933 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 419/846] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:18:41.743778 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348f487-fd5b-4ee6-8ac8-6e9e07ffc85d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:41.764633 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 428/847] 10.222.0.20 () {56 vars in 1149 bytes} [Sat Aug 6 12:18:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:41.766418 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14932bb2-5317-4b82-ae60-95a6c56a7737 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:41.769852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:41.849852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348f487-fd5b-4ee6-8ac8-6e9e07ffc85d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:41.865890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:41.874764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14932bb2-5317-4b82-ae60-95a6c56a7737 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:41.884677 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ab536-195f-42b5-b75c-5ccbaf827751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:41.885368 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ab536-195f-42b5-b75c-5ccbaf827751 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:41.885571 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ab536-195f-42b5-b75c-5ccbaf827751 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:41.885846 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ab536-195f-42b5-b75c-5ccbaf827751 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:41.886532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ab536-195f-42b5-b75c-5ccbaf827751 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:41.886827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ab536-195f-42b5-b75c-5ccbaf827751 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:41.888240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ab536-195f-42b5-b75c-5ccbaf827751 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:41.900110 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 420/848] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:41.900938 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ab536-195f-42b5-b75c-5ccbaf827751 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:41.917347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55f95b-b612-438b-9e6f-f703908187c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:41.917988 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55f95b-b612-438b-9e6f-f703908187c1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:41.918279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55f95b-b612-438b-9e6f-f703908187c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:41.918551 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55f95b-b612-438b-9e6f-f703908187c1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:41.919181 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55f95b-b612-438b-9e6f-f703908187c1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:41.920570 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55f95b-b612-438b-9e6f-f703908187c1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:41.929574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-9e55f95b-b612-438b-9e6f-f703908187c1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 12:18:41.929574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:41.929574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:18:41.929574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:18:41.929574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:18:41.929574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:18:41.929574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:18:41.929574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:41.929574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 12:18:41.929574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:41.929574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:18:41.929574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:41.929574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:41.929574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:41.929574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:41.929574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:41.929574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:41.930592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:41.930592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:41.930592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:41.930592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:41.930592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:41.930592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:41.930592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:18:41.930592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:18:41.930592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:18:41.930592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:18:41.930592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:41.930592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:41.930592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:41.930592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:41.930592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:41.930592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:41.936875 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:41.936875 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:41.936875 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:41.936875 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:41.936875 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:41.936875 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:41.936875 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:41.936875 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:41.936875 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:41.936875 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:41.936875 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:18:41.936875 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:41.936875 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 12:18:41.936875 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:41.936875 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 429/849] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Aug 6 12:18:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:41.937686 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55f95b-b612-438b-9e6f-f703908187c1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:41.995519 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:42.014040 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ab536-195f-42b5-b75c-5ccbaf827751 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:42.048796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24a190-4fae-42c9-a435-a49b7bd12507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:42.049423 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24a190-4fae-42c9-a435-a49b7bd12507 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:42.049706 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24a190-4fae-42c9-a435-a49b7bd12507 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:42.050073 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24a190-4fae-42c9-a435-a49b7bd12507 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:42.054016 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55f95b-b612-438b-9e6f-f703908187c1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:42.100339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df889f5b-14e5-435f-b94d-96feafddb175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:42.100339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df889f5b-14e5-435f-b94d-96feafddb175 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:42.100339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df889f5b-14e5-435f-b94d-96feafddb175 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:42.100339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df889f5b-14e5-435f-b94d-96feafddb175 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:42.102877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df889f5b-14e5-435f-b94d-96feafddb175 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:42.104338 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df889f5b-14e5-435f-b94d-96feafddb175 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:42.116509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-df889f5b-14e5-435f-b94d-96feafddb175 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 06 12:18:42.116509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:42.116509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:42.116509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:42.116509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:42.116509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:42.116509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:42.116509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:42.116509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:42.116509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:42.116509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:42.116509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:42.116509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:42.116509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:42.116509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:42.116509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:42.118115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:42.118115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:42.118115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:42.118115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:42.118115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:42.118115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:42.118115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:42.118115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:42.118115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:42.118115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:42.118115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:42.118115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:42.118115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:42.118115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:42.118115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:42.118115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:42.118115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:42.118115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:42.119073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:42.119073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:42.119073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:42.119073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:42.119073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:42.119073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:42.119073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:42.119073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:42.119073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 06 12:18:42.119073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:42.119073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 430/850] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:18:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:42.119073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df889f5b-14e5-435f-b94d-96feafddb175 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:42.131513 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce24a190-4fae-42c9-a435-a49b7bd12507 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:42.133080 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce24a190-4fae-42c9-a435-a49b7bd12507 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:42.141743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:42.224098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:42.240019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df889f5b-14e5-435f-b94d-96feafddb175 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:42.243191 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 421/851] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:18:41 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 12:18:42.244015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24a190-4fae-42c9-a435-a49b7bd12507 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:42.284448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56fb75b-3c81-41d3-a3e0-c68cc14c041a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:42.285068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56fb75b-3c81-41d3-a3e0-c68cc14c041a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:42.285340 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56fb75b-3c81-41d3-a3e0-c68cc14c041a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:42.285616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56fb75b-3c81-41d3-a3e0-c68cc14c041a None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:42.286286 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56fb75b-3c81-41d3-a3e0-c68cc14c041a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:42.286585 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56fb75b-3c81-41d3-a3e0-c68cc14c041a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:42.287937 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56fb75b-3c81-41d3-a3e0-c68cc14c041a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:42.297721 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 431/852] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Aug 6 12:18:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:42.297721 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56fb75b-3c81-41d3-a3e0-c68cc14c041a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:42.349535 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24a190-4fae-42c9-a435-a49b7bd12507 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:42.379924 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:42.385347 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063455ea-9ffe-4866-a3a4-e07c4f7ac54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:42.385952 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063455ea-9ffe-4866-a3a4-e07c4f7ac54b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:42.386228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063455ea-9ffe-4866-a3a4-e07c4f7ac54b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:42.386489 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063455ea-9ffe-4866-a3a4-e07c4f7ac54b None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:42.387273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063455ea-9ffe-4866-a3a4-e07c4f7ac54b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:42.387581 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063455ea-9ffe-4866-a3a4-e07c4f7ac54b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:42.389024 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063455ea-9ffe-4866-a3a4-e07c4f7ac54b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:42.400647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56fb75b-3c81-41d3-a3e0-c68cc14c041a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:42.406988 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 422/853] 10.222.0.20 () {56 vars in 1135 bytes} [Sat Aug 6 12:18:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:42.407562 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063455ea-9ffe-4866-a3a4-e07c4f7ac54b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:42.440921 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3d316-1c89-43bf-9dd8-b3a56253c38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:42.441563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3d316-1c89-43bf-9dd8-b3a56253c38c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:42.441844 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3d316-1c89-43bf-9dd8-b3a56253c38c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:42.442119 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3d316-1c89-43bf-9dd8-b3a56253c38c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:42.450615 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:42.453951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb3d316-1c89-43bf-9dd8-b3a56253c38c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:42.455426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb3d316-1c89-43bf-9dd8-b3a56253c38c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:42.472288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-ecb3d316-1c89-43bf-9dd8-b3a56253c38c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:42.472288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:42.472288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:42.472288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:42.472288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:42.472288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:42.472288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:42.472288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:42.472288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:42.472288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:42.472288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:42.472288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:42.472288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:42.472288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:42.472288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:42.473395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:42.473395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:42.473395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:42.473395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:42.473395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:42.473395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:42.473395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:42.473395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:42.473395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:42.473395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:42.473395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:42.473395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:42.473395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:42.473395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:42.473395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:42.473395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:42.473395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:42.474427 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:42.474427 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:42.474427 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:42.474427 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:42.474427 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 432/854] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:42.474767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3d316-1c89-43bf-9dd8-b3a56253c38c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:42.518029 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063455ea-9ffe-4866-a3a4-e07c4f7ac54b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:42.556653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640d6fb-d465-4fd5-8cbc-1287e0e098bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:42.557469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640d6fb-d465-4fd5-8cbc-1287e0e098bc None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:42.557882 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640d6fb-d465-4fd5-8cbc-1287e0e098bc None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:42.558291 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640d6fb-d465-4fd5-8cbc-1287e0e098bc None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:42.559318 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640d6fb-d465-4fd5-8cbc-1287e0e098bc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:42.559782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640d6fb-d465-4fd5-8cbc-1287e0e098bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:42.561945 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640d6fb-d465-4fd5-8cbc-1287e0e098bc None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:42.578515 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 423/855] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Aug 6 12:18:42 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:42.578515 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640d6fb-d465-4fd5-8cbc-1287e0e098bc None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:42.604024 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3d316-1c89-43bf-9dd8-b3a56253c38c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:42.645525 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2cc39-5d7a-43ab-9138-40789d898bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:42.646163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2cc39-5d7a-43ab-9138-40789d898bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:42.646446 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2cc39-5d7a-43ab-9138-40789d898bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:42.646721 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2cc39-5d7a-43ab-9138-40789d898bf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:42.660688 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:42.703586 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640d6fb-d465-4fd5-8cbc-1287e0e098bc None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:42.712529 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:42.735503 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2cc39-5d7a-43ab-9138-40789d898bf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:42.737115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2cc39-5d7a-43ab-9138-40789d898bf9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:42.747514 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b5fc6-a90e-4245-9660-8cea7faee75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:42.748358 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b5fc6-a90e-4245-9660-8cea7faee75b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:42.748816 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b5fc6-a90e-4245-9660-8cea7faee75b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:42.749071 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b5fc6-a90e-4245-9660-8cea7faee75b None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:42.749945 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b5fc6-a90e-4245-9660-8cea7faee75b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:42.750263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b5fc6-a90e-4245-9660-8cea7faee75b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:42.752514 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b5fc6-a90e-4245-9660-8cea7faee75b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:42.766725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 424/856] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:42.772483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b5fc6-a90e-4245-9660-8cea7faee75b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:42.859957 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 433/857] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:18:42 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 12:18:42.865141 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2cc39-5d7a-43ab-9138-40789d898bf9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:42.901541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:42.906985 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b5fc6-a90e-4245-9660-8cea7faee75b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:42.957301 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf656348-14ae-4f35-ab86-425da55217e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:42.958553 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf656348-14ae-4f35-ab86-425da55217e3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:42.959073 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf656348-14ae-4f35-ab86-425da55217e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:42.959926 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf656348-14ae-4f35-ab86-425da55217e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:42.970505 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf656348-14ae-4f35-ab86-425da55217e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:42.972773 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf656348-14ae-4f35-ab86-425da55217e3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:42.976093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2cc39-5d7a-43ab-9138-40789d898bf9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:42.988653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-bf656348-14ae-4f35-ab86-425da55217e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:42.988653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:42.988653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:42.988653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:42.988653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:42.988653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:42.988653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:42.988653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:42.988653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:42.988653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:42.988653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:42.988653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:42.988653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:42.988653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:42.988653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:42.989763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:42.989763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:42.989763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:42.989763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:42.989763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:42.989763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:42.989763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:42.989763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:42.989763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:42.989763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:42.989763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:42.989763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:42.989763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:42.989763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:42.989763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:42.989763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:42.989763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:42.990873 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:42.990873 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:42.990873 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:42.990873 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:42.990873 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:42.990873 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:42.990873 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:42.990873 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:42.994080 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 425/858] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:42.999724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf656348-14ae-4f35-ab86-425da55217e3 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:43.023305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd501ae-9c98-481c-8579-dfa7a732cf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:43.024194 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd501ae-9c98-481c-8579-dfa7a732cf58 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:43.024583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd501ae-9c98-481c-8579-dfa7a732cf58 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:43.024979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd501ae-9c98-481c-8579-dfa7a732cf58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:43.037054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:43.037742 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd501ae-9c98-481c-8579-dfa7a732cf58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:43.039925 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd501ae-9c98-481c-8579-dfa7a732cf58 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:43.055871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-2dd501ae-9c98-481c-8579-dfa7a732cf58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:43.055871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:43.055871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:43.055871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:43.055871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:43.055871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:43.055871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:43.055871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:43.055871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:43.055871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:43.055871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:43.055871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:43.055871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:43.055871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:43.055871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:43.056863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:43.056863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:43.056863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:43.056863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:43.056863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:43.056863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:43.056863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:43.056863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:43.056863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:43.056863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:43.056863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:43.056863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:43.056863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:43.056863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:43.056863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:43.056863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:43.056863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:43.057832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:43.057832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:43.057832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:43.057832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:43.060451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 434/859] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:43.061999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82073ecf-b613-4d0c-b7ef-b70384d56c87 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:43.062430 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82073ecf-b613-4d0c-b7ef-b70384d56c87 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:43.062878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82073ecf-b613-4d0c-b7ef-b70384d56c87 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:43.064329 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 435/860] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:43.064897 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82073ecf-b613-4d0c-b7ef-b70384d56c87 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:43.122111 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf656348-14ae-4f35-ab86-425da55217e3 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:43.158588 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ff8af-61ee-47e7-b30c-914821e5fb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:43.159472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ff8af-61ee-47e7-b30c-914821e5fb0d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:43.159889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ff8af-61ee-47e7-b30c-914821e5fb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:43.160323 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ff8af-61ee-47e7-b30c-914821e5fb0d None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:43.161200 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ff8af-61ee-47e7-b30c-914821e5fb0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:43.161642 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ff8af-61ee-47e7-b30c-914821e5fb0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:43.163242 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ff8af-61ee-47e7-b30c-914821e5fb0d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:43.165117 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:43.180886 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 426/861] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:43.182149 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c22d35-947b-47da-987a-56cfa704cecf None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:43.182554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c22d35-947b-47da-987a-56cfa704cecf None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:43.182942 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c22d35-947b-47da-987a-56cfa704cecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:43.183919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82073ecf-b613-4d0c-b7ef-b70384d56c87 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:43.200173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-61c22d35-947b-47da-987a-56cfa704cecf None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:43.201619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-61c22d35-947b-47da-987a-56cfa704cecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:43.219294 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f24f94-d522-42a5-b124-398b8b014103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:43.219910 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f24f94-d522-42a5-b124-398b8b014103 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:43.220195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f24f94-d522-42a5-b124-398b8b014103 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:43.220498 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f24f94-d522-42a5-b124-398b8b014103 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:43.221197 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f24f94-d522-42a5-b124-398b8b014103 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:43.221500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f24f94-d522-42a5-b124-398b8b014103 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:43.222858 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f24f94-d522-42a5-b124-398b8b014103 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:43.235476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 436/862] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:43.236326 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f24f94-d522-42a5-b124-398b8b014103 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:43.280354 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 427/863] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:18:43.280354 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c22d35-947b-47da-987a-56cfa704cecf None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:43.306202 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:43.343963 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f24f94-d522-42a5-b124-398b8b014103 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:43.380510 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785dd7e-e945-4f80-9d7a-f38276af431e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:43.381150 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785dd7e-e945-4f80-9d7a-f38276af431e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:43.381426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785dd7e-e945-4f80-9d7a-f38276af431e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:43.381692 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785dd7e-e945-4f80-9d7a-f38276af431e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:43.382465 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6785dd7e-e945-4f80-9d7a-f38276af431e None admin] RBAC: Adding query filter params (role.id=785cca921bad4984a86e45ce6c5601c2, scope.project.id=13fba33c46cf40bbb15a08c690b4c98c, user.id=c810b52785b94c0cab4fd9f0d6e65825) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:43.382796 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6785dd7e-e945-4f80-9d7a-f38276af431e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:43.385008 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6785dd7e-e945-4f80-9d7a-f38276af431e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:43.389897 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c22d35-947b-47da-987a-56cfa704cecf None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:43.400462 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 437/864] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:18:43 2022] GET /identity/v3/role_assignments?user.id=c810b52785b94c0cab4fd9f0d6e65825&scope.project.id=13fba33c46cf40bbb15a08c690b4c98c&role.id=785cca921bad4984a86e45ce6c5601c2 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:43.400462 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785dd7e-e945-4f80-9d7a-f38276af431e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:43.420119 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:43.426925 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646e9bb-8354-4828-baee-c38a2c7b80d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:43.427573 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646e9bb-8354-4828-baee-c38a2c7b80d0 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:43.427848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646e9bb-8354-4828-baee-c38a2c7b80d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:43.428117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646e9bb-8354-4828-baee-c38a2c7b80d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:43.437500 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646e9bb-8354-4828-baee-c38a2c7b80d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:43.439034 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646e9bb-8354-4828-baee-c38a2c7b80d0 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:43.451447 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-7646e9bb-8354-4828-baee-c38a2c7b80d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:43.451447 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:43.451447 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:43.451447 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:43.451447 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:43.451447 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:43.451447 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:43.451447 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:43.451447 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:43.451447 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:43.451447 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:43.451447 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:43.451447 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:43.451447 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:43.451447 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:43.452509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:43.452509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:43.452509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:43.452509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:43.452509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:43.452509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:43.452509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:43.452509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:43.452509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:43.452509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:43.452509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:43.452509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:43.452509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:43.452509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:43.452509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:43.452509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:43.452509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:43.453387 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:43.453387 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:43.453387 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:43.453387 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:43.453387 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:43.453387 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:43.453387 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:43.453387 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:43.453387 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 428/865] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:43.459890 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646e9bb-8354-4828-baee-c38a2c7b80d0 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:43.537199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785dd7e-e945-4f80-9d7a-f38276af431e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:43.560953 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:43.575281 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8dbe2f-a6c6-46f1-b9d7-d533d71f0462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:43.576146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8dbe2f-a6c6-46f1-b9d7-d533d71f0462 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:43.576456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8dbe2f-a6c6-46f1-b9d7-d533d71f0462 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:43.576732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8dbe2f-a6c6-46f1-b9d7-d533d71f0462 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:43.577356 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8dbe2f-a6c6-46f1-b9d7-d533d71f0462 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:43.578689 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8dbe2f-a6c6-46f1-b9d7-d533d71f0462 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:43.587709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-0a8dbe2f-a6c6-46f1-b9d7-d533d71f0462 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 12:18:43.587709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:43.587709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:18:43.587709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:18:43.587709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:18:43.587709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:18:43.587709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:18:43.587709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:43.587709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 12:18:43.587709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:43.587709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:18:43.587709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:43.587709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:43.587709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:43.587709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:43.587709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:43.588702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:43.588702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:43.588702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:43.588702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:43.588702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:43.588702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:43.588702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:43.588702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:18:43.588702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:18:43.588702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:18:43.588702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:18:43.588702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:43.588702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:43.588702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:43.588702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:43.588702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:43.589520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:43.589520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:43.589520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:43.589520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:43.589520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:43.589520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:43.589520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:43.589520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:43.589520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:43.589520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:43.589520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:43.589520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:18:43.589520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:43.589520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 12:18:43.589520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:43.590306 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 438/866] 10.222.0.20 () {56 vars in 1132 bytes} [Sat Aug 6 12:18:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:43.592893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646e9bb-8354-4828-baee-c38a2c7b80d0 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:43.595621 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8dbe2f-a6c6-46f1-b9d7-d533d71f0462 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:43.638125 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e610d21-9dbc-4e17-8bd7-99c76eb000e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:43.638818 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e610d21-9dbc-4e17-8bd7-99c76eb000e7 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:43.639210 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e610d21-9dbc-4e17-8bd7-99c76eb000e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:43.639579 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e610d21-9dbc-4e17-8bd7-99c76eb000e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:43.640528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e610d21-9dbc-4e17-8bd7-99c76eb000e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:43.640930 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e610d21-9dbc-4e17-8bd7-99c76eb000e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:43.642455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e610d21-9dbc-4e17-8bd7-99c76eb000e7 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:43.656897 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 429/867] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:43.662685 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e610d21-9dbc-4e17-8bd7-99c76eb000e7 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:43.694967 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:43.732446 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8dbe2f-a6c6-46f1-b9d7-d533d71f0462 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:43.782753 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e610d21-9dbc-4e17-8bd7-99c76eb000e7 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:43.784020 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf9cec-30ce-42c9-90f7-4fc1d2c4aff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:43.785209 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf9cec-30ce-42c9-90f7-4fc1d2c4aff8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:43.785695 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf9cec-30ce-42c9-90f7-4fc1d2c4aff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:43.786168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf9cec-30ce-42c9-90f7-4fc1d2c4aff8 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:43.787061 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bf9cec-30ce-42c9-90f7-4fc1d2c4aff8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:43.787561 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bf9cec-30ce-42c9-90f7-4fc1d2c4aff8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:43.789822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bf9cec-30ce-42c9-90f7-4fc1d2c4aff8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:43.795510 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 439/868] 10.222.0.20 () {56 vars in 1126 bytes} [Sat Aug 6 12:18:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:43.805579 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf9cec-30ce-42c9-90f7-4fc1d2c4aff8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:43.819320 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c123a57-2204-4433-b9f6-8cab3c78efe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:43.819929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c123a57-2204-4433-b9f6-8cab3c78efe6 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:43.820202 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c123a57-2204-4433-b9f6-8cab3c78efe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:43.820585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c123a57-2204-4433-b9f6-8cab3c78efe6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:43.821370 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c123a57-2204-4433-b9f6-8cab3c78efe6 None admin] RBAC: Adding query filter params (role.id=144c8425688b409384e1e8ed8279d88e, scope.project.id=13fba33c46cf40bbb15a08c690b4c98c, user.id=2efc792a2655439f8b8070010059b6bb) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:43.821731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c123a57-2204-4433-b9f6-8cab3c78efe6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:43.823256 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c123a57-2204-4433-b9f6-8cab3c78efe6 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:43.826677 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:43.831122 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 430/869] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:18:43 2022] GET /identity/v3/role_assignments?user.id=2efc792a2655439f8b8070010059b6bb&scope.project.id=13fba33c46cf40bbb15a08c690b4c98c&role.id=144c8425688b409384e1e8ed8279d88e => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:43.929123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf9cec-30ce-42c9-90f7-4fc1d2c4aff8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:43.967318 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e057e-ba83-486f-8072-d6d8d844d48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:43.968015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e057e-ba83-486f-8072-d6d8d844d48e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:43.968320 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e057e-ba83-486f-8072-d6d8d844d48e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:43.968602 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e057e-ba83-486f-8072-d6d8d844d48e None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:43.969349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e057e-ba83-486f-8072-d6d8d844d48e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:43.969651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e057e-ba83-486f-8072-d6d8d844d48e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:43.971039 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e057e-ba83-486f-8072-d6d8d844d48e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:43.977376 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 440/870] 10.222.0.20 () {56 vars in 1126 bytes} [Sat Aug 6 12:18:43 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:44.652629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f960020-02c2-4f80-a8f1-76926fb8022b None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:44.653185 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f960020-02c2-4f80-a8f1-76926fb8022b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:44.653185 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f960020-02c2-4f80-a8f1-76926fb8022b None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:44.654775 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 431/871] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:44.660119 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9e12b-1c52-4fab-95e5-a5fe85bacea7 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:44.660436 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9e12b-1c52-4fab-95e5-a5fe85bacea7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:44.660716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9e12b-1c52-4fab-95e5-a5fe85bacea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:44.667649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:44.681750 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-7af9e12b-1c52-4fab-95e5-a5fe85bacea7 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:44.682943 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7af9e12b-1c52-4fab-95e5-a5fe85bacea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:44.755674 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 441/872] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 12:18:44.778135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f960020-02c2-4f80-a8f1-76926fb8022b None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:44.883644 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f960020-02c2-4f80-a8f1-76926fb8022b None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:44.918135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f42c81-b9c5-441a-8bcc-bc0b93d3f214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:44.918680 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f42c81-b9c5-441a-8bcc-bc0b93d3f214 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:44.918958 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f42c81-b9c5-441a-8bcc-bc0b93d3f214 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:44.919232 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f42c81-b9c5-441a-8bcc-bc0b93d3f214 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:44.919935 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f42c81-b9c5-441a-8bcc-bc0b93d3f214 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:44.921326 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f42c81-b9c5-441a-8bcc-bc0b93d3f214 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:44.937877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 432/873] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Aug 6 12:18:44 2022] POST /identity/v3/services => generated 251 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:18:45.620871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe86ca-ad8d-43b8-afcc-a6c6cc62e1d8 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:45.621430 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe86ca-ad8d-43b8-afcc-a6c6cc62e1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:45.621430 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe86ca-ad8d-43b8-afcc-a6c6cc62e1d8 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:45.623060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 442/874] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:45.628461 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6dfc8-25db-4f1c-95c4-afb43452bf2a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:45.628752 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6dfc8-25db-4f1c-95c4-afb43452bf2a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:45.629029 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6dfc8-25db-4f1c-95c4-afb43452bf2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:45.648089 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-d4d6dfc8-25db-4f1c-95c4-afb43452bf2a None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:45.649832 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d4d6dfc8-25db-4f1c-95c4-afb43452bf2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:45.707358 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:45.728184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 433/875] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:45 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:18:45.751490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe86ca-ad8d-43b8-afcc-a6c6cc62e1d8 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:45.857289 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe86ca-ad8d-43b8-afcc-a6c6cc62e1d8 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:45.887749 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:45.893292 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f3798-aed2-4a19-b229-522e70f52c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:45.893934 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f3798-aed2-4a19-b229-522e70f52c59 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:45.894213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f3798-aed2-4a19-b229-522e70f52c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:45.894483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f3798-aed2-4a19-b229-522e70f52c59 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:45.905464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f3798-aed2-4a19-b229-522e70f52c59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:45.910388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-fc7f3798-aed2-4a19-b229-522e70f52c59 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 12:18:45.910388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:45.910388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:45.910388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:45.910388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:45.910388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:45.910388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:45.910388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:45.910388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:45.910388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:45.910388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:45.910388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:45.910388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:45.910388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:45.910388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:45.910388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:45.912269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:45.912269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:45.912269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:45.912269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:45.912269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:45.912269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:45.912269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:45.912269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:45.912269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:45.912269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:45.912269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:45.912269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:45.912269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:45.912269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:45.912269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:45.912269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:45.913374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:45.913374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:45.913374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:45.913374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:45.913374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:45.913374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:45.913374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:45.913374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:45.913374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:45.913374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:45.913374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 12:18:45.913374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:45.913374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 443/876] 10.222.0.20 () {56 vars in 1120 bytes} [Sat Aug 6 12:18:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:45.917947 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6dfc8-25db-4f1c-95c4-afb43452bf2a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:45.949629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57db235-8c97-48ae-b66a-c2a860da8e46 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:45.949629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57db235-8c97-48ae-b66a-c2a860da8e46 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:45.949629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57db235-8c97-48ae-b66a-c2a860da8e46 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:45.951278 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 444/877] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:18:45.956147 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b1603-a4c3-4098-93bb-691912ca37de None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:45.956719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b1603-a4c3-4098-93bb-691912ca37de None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:45.957211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b1603-a4c3-4098-93bb-691912ca37de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:45.976278 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d83b1603-a4c3-4098-93bb-691912ca37de None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:45.977886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d83b1603-a4c3-4098-93bb-691912ca37de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:46.030000 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6dfc8-25db-4f1c-95c4-afb43452bf2a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:46.062788 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 445/878] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:45 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 12:18:46.064498 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa6b71-2252-452b-ae2e-11068b490eb9 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:46.064782 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa6b71-2252-452b-ae2e-11068b490eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:46.065053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa6b71-2252-452b-ae2e-11068b490eb9 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:46.066471 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 446/879] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:46.071430 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbdd5a-62e6-4b8c-a627-a76e32cd7617 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:46.071713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbdd5a-62e6-4b8c-a627-a76e32cd7617 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:46.071989 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbdd5a-62e6-4b8c-a627-a76e32cd7617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:46.072336 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087efa6-9799-472f-a6b3-fffac67f05e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:46.073226 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087efa6-9799-472f-a6b3-fffac67f05e4 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:46.073693 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087efa6-9799-472f-a6b3-fffac67f05e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:46.074152 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087efa6-9799-472f-a6b3-fffac67f05e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:46.075147 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c087efa6-9799-472f-a6b3-fffac67f05e4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:46.075652 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c087efa6-9799-472f-a6b3-fffac67f05e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:46.077283 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c087efa6-9799-472f-a6b3-fffac67f05e4 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:46.086745 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 434/880] 10.222.0.20 () {56 vars in 1115 bytes} [Sat Aug 6 12:18:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:46.087895 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087efa6-9799-472f-a6b3-fffac67f05e4 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:46.092366 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0edbdd5a-62e6-4b8c-a627-a76e32cd7617 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:46.093623 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0edbdd5a-62e6-4b8c-a627-a76e32cd7617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:46.146845 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:46.165150 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 447/881] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:46 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 12:18:46.166316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbdd5a-62e6-4b8c-a627-a76e32cd7617 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:46.212430 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087efa6-9799-472f-a6b3-fffac67f05e4 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:46.227095 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:46.248822 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6c913-5342-4eb3-907f-0e72e1746443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:46.249458 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6c913-5342-4eb3-907f-0e72e1746443 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:46.249742 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6c913-5342-4eb3-907f-0e72e1746443 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:46.250149 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6c913-5342-4eb3-907f-0e72e1746443 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:46.250778 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6c913-5342-4eb3-907f-0e72e1746443 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:46.252144 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6c913-5342-4eb3-907f-0e72e1746443 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:46.261381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-6ab6c913-5342-4eb3-907f-0e72e1746443 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 12:18:46.261381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:46.261381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:18:46.261381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:18:46.261381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:18:46.261381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:18:46.261381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:18:46.261381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:46.261381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 12:18:46.261381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:46.261381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:18:46.261381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:46.261381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:46.261381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:46.261381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:46.261381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:46.261381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:46.262372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:46.262372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:46.262372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:46.262372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:46.262372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:46.262372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:46.262372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:18:46.262372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:18:46.262372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:18:46.262372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:18:46.262372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:46.262372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:46.262372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:46.262372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:46.262372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:46.262372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:46.263201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:46.263201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:46.263201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:46.263201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:46.263201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:46.263201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:46.263201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:46.263201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:46.263201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:46.263201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:46.263201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:18:46.263201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:46.263201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 12:18:46.263201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:46.263201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 435/882] 10.222.0.20 () {56 vars in 1126 bytes} [Sat Aug 6 12:18:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:46.263982 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6c913-5342-4eb3-907f-0e72e1746443 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:46.283057 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbdd5a-62e6-4b8c-a627-a76e32cd7617 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:46.318460 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f87941-d7aa-442c-90bf-a4a77af5a92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:46.319098 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f87941-d7aa-442c-90bf-a4a77af5a92b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:46.319378 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f87941-d7aa-442c-90bf-a4a77af5a92b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:46.319649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f87941-d7aa-442c-90bf-a4a77af5a92b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:46.330497 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f87941-d7aa-442c-90bf-a4a77af5a92b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:46.331921 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f87941-d7aa-442c-90bf-a4a77af5a92b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:46.347009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-42f87941-d7aa-442c-90bf-a4a77af5a92b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:46.347009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:46.347009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:46.347009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:46.347009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:46.347009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:46.347009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:46.347009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:46.347009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:46.347009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:46.347009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:46.347009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:46.347009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:46.347009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:46.347009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:46.348019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:46.348019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:46.348019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:46.348019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:46.348019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:46.348019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:46.348019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:46.348019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:46.348019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:46.348019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:46.348019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:46.348019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:46.348019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:46.348019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:46.348019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:46.348019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:46.348019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:46.349110 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:46.349110 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:46.349110 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:46.349110 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:46.352841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 448/883] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:46.353722 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f87941-d7aa-442c-90bf-a4a77af5a92b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:46.374870 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6c913-5342-4eb3-907f-0e72e1746443 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:46.387545 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:46.412355 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7e8c0-abb3-4e4f-851a-7002ba6fa2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:46.413054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7e8c0-abb3-4e4f-851a-7002ba6fa2d4 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:46.413174 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7e8c0-abb3-4e4f-851a-7002ba6fa2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:46.413449 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7e8c0-abb3-4e4f-851a-7002ba6fa2d4 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:46.414193 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7e8c0-abb3-4e4f-851a-7002ba6fa2d4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:46.415565 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7e8c0-abb3-4e4f-851a-7002ba6fa2d4 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:46.436871 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 436/884] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Aug 6 12:18:46 2022] POST /identity/v3/services => generated 269 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:18:46.436871 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7e8c0-abb3-4e4f-851a-7002ba6fa2d4 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:46.468038 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f87941-d7aa-442c-90bf-a4a77af5a92b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:46.471936 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:46.522028 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa9bb9-ca5f-47a1-a0d1-c8e1483d8eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:46.523078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa9bb9-ca5f-47a1-a0d1-c8e1483d8eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:46.524088 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa9bb9-ca5f-47a1-a0d1-c8e1483d8eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:46.524552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa9bb9-ca5f-47a1-a0d1-c8e1483d8eb6 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:46.525501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa9bb9-ca5f-47a1-a0d1-c8e1483d8eb6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:46.525957 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa9bb9-ca5f-47a1-a0d1-c8e1483d8eb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:46.527524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa9bb9-ca5f-47a1-a0d1-c8e1483d8eb6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:46.538028 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 449/885] 10.222.0.20 () {56 vars in 1122 bytes} [Sat Aug 6 12:18:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:46.539317 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa9bb9-ca5f-47a1-a0d1-c8e1483d8eb6 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:46.595900 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7e8c0-abb3-4e4f-851a-7002ba6fa2d4 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:46.641037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5541c2-29f3-4c3d-8760-cfeca5411bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:46.641944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5541c2-29f3-4c3d-8760-cfeca5411bfa None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:46.642277 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5541c2-29f3-4c3d-8760-cfeca5411bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:46.642554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5541c2-29f3-4c3d-8760-cfeca5411bfa None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:46.643246 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5541c2-29f3-4c3d-8760-cfeca5411bfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:46.643544 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5541c2-29f3-4c3d-8760-cfeca5411bfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:46.645182 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5541c2-29f3-4c3d-8760-cfeca5411bfa None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:46.665721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 437/886] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:46.667379 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5541c2-29f3-4c3d-8760-cfeca5411bfa None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:46.670171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa9bb9-ca5f-47a1-a0d1-c8e1483d8eb6 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:46.701296 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:46.714066 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7e30a-9a6e-4060-a458-123aed66cab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:46.715080 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7e30a-9a6e-4060-a458-123aed66cab9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:46.715610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7e30a-9a6e-4060-a458-123aed66cab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:46.716083 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7e30a-9a6e-4060-a458-123aed66cab9 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:46.717157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c7e30a-9a6e-4060-a458-123aed66cab9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:46.717662 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c7e30a-9a6e-4060-a458-123aed66cab9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:46.719393 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c7e30a-9a6e-4060-a458-123aed66cab9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:46.725355 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 450/887] 10.222.0.20 () {56 vars in 1122 bytes} [Sat Aug 6 12:18:46 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:46.788070 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5541c2-29f3-4c3d-8760-cfeca5411bfa None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:46.827915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045ba96-b7ae-48db-97a1-84cb6c760baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:46.828980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045ba96-b7ae-48db-97a1-84cb6c760baa None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:46.829494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045ba96-b7ae-48db-97a1-84cb6c760baa None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:46.829972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045ba96-b7ae-48db-97a1-84cb6c760baa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:46.839259 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0045ba96-b7ae-48db-97a1-84cb6c760baa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:46.841164 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0045ba96-b7ae-48db-97a1-84cb6c760baa None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:46.857459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-0045ba96-b7ae-48db-97a1-84cb6c760baa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 12:18:46.857459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:46.857459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:46.857459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:46.857459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:46.857459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:46.857459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:46.857459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:46.857459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:46.857459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:46.857459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:46.857459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:46.857459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:46.857459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:46.857459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:46.857459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:46.858482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:46.858482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:46.858482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:46.858482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:46.858482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:46.858482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:46.858482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:46.858482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:46.858482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:46.858482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:46.858482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:46.858482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:46.858482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:46.858482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:46.858482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:46.858482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:46.858482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:46.858482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:46.859453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:46.859453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:46.859453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:46.859453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:46.859453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:46.859453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:46.859453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:46.859453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:46.859453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 12:18:46.859453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:46.865873 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 438/888] 10.222.0.20 () {56 vars in 1117 bytes} [Sat Aug 6 12:18:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:46.869412 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7e30a-9a6e-4060-a458-123aed66cab9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:46.901377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:46.983663 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7e30a-9a6e-4060-a458-123aed66cab9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:47.023537 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d9ccb-03a3-4f9d-941d-0519e4a5bb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:47.024421 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d9ccb-03a3-4f9d-941d-0519e4a5bb3d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:47.024869 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d9ccb-03a3-4f9d-941d-0519e4a5bb3d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:47.025287 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d9ccb-03a3-4f9d-941d-0519e4a5bb3d None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:47.026236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d9ccb-03a3-4f9d-941d-0519e4a5bb3d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:47.026687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d9ccb-03a3-4f9d-941d-0519e4a5bb3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:47.028356 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d9ccb-03a3-4f9d-941d-0519e4a5bb3d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:47.051750 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 451/889] 10.222.0.20 () {56 vars in 1149 bytes} [Sat Aug 6 12:18:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:47.053326 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045ba96-b7ae-48db-97a1-84cb6c760baa None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:47.090998 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f70c5d-16a8-4a2d-b60f-9357ce5f36d3 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:47.091545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f70c5d-16a8-4a2d-b60f-9357ce5f36d3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:47.092020 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f70c5d-16a8-4a2d-b60f-9357ce5f36d3 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:47.093940 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 452/890] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:47.099009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ee41a-7951-4621-a183-94caa5ab8a88 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:47.099407 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ee41a-7951-4621-a183-94caa5ab8a88 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:47.099786 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ee41a-7951-4621-a183-94caa5ab8a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:47.118556 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3d8ee41a-7951-4621-a183-94caa5ab8a88 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:47.119889 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3d8ee41a-7951-4621-a183-94caa5ab8a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:47.166763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:47.181259 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045ba96-b7ae-48db-97a1-84cb6c760baa None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:47.200546 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 453/891] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 12:18:47.215847 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ee41a-7951-4621-a183-94caa5ab8a88 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:47.221423 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198a851-5eb4-4050-854f-2c9111e23ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:47.222324 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198a851-5eb4-4050-854f-2c9111e23ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:47.222750 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198a851-5eb4-4050-854f-2c9111e23ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:47.223279 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198a851-5eb4-4050-854f-2c9111e23ea5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:47.235029 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:47.314992 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198a851-5eb4-4050-854f-2c9111e23ea5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:47.316707 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198a851-5eb4-4050-854f-2c9111e23ea5 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:47.345387 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ee41a-7951-4621-a183-94caa5ab8a88 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:47.387098 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8b06c-b382-456f-bf58-ca08f87556c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:47.387862 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8b06c-b382-456f-bf58-ca08f87556c1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:47.388215 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8b06c-b382-456f-bf58-ca08f87556c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:47.388591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8b06c-b382-456f-bf58-ca08f87556c1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:47.389378 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8b06c-b382-456f-bf58-ca08f87556c1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:47.390926 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8b06c-b382-456f-bf58-ca08f87556c1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:47.400448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-00e8b06c-b382-456f-bf58-ca08f87556c1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 12:18:47.400448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:47.400448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:18:47.400448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:18:47.400448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:18:47.400448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:18:47.400448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:18:47.400448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:47.400448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 12:18:47.400448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:47.400448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:18:47.400448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:47.400448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:47.400448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:47.400448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:47.400448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:47.400448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:47.401399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:47.401399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:47.401399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:47.401399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:47.401399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:47.401399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:47.401399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:18:47.401399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:18:47.401399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:18:47.401399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:18:47.401399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:47.401399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:47.401399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:47.401399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:47.401399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:47.401399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:47.402229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:47.402229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:47.402229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:47.402229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:47.402229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:47.402229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:47.402229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:47.402229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:47.402229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:47.402229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:47.402229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:18:47.402229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:47.402229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 12:18:47.402229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:47.406635 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 454/892] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Aug 6 12:18:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:47.407461 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8b06c-b382-456f-bf58-ca08f87556c1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:47.418730 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:47.432490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 439/893] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:18:47 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 12:18:47.440664 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198a851-5eb4-4050-854f-2c9111e23ea5 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:47.508805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:47.521221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8b06c-b382-456f-bf58-ca08f87556c1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:47.549175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198a851-5eb4-4050-854f-2c9111e23ea5 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:47.560363 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c3bfd-882e-442b-addd-ad2b915e00c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:47.561182 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c3bfd-882e-442b-addd-ad2b915e00c3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:47.561648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c3bfd-882e-442b-addd-ad2b915e00c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:47.562107 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c3bfd-882e-442b-addd-ad2b915e00c3 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:47.563091 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c3bfd-882e-442b-addd-ad2b915e00c3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:47.563571 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c3bfd-882e-442b-addd-ad2b915e00c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:47.565236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c3bfd-882e-442b-addd-ad2b915e00c3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:47.571625 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 455/894] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Aug 6 12:18:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:47.580423 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c3bfd-882e-442b-addd-ad2b915e00c3 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:47.585653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e49763-23d2-47b3-841c-fd3949f8e4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:47.586861 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e49763-23d2-47b3-841c-fd3949f8e4ee None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:47.587141 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e49763-23d2-47b3-841c-fd3949f8e4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:47.587414 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e49763-23d2-47b3-841c-fd3949f8e4ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:47.598173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e49763-23d2-47b3-841c-fd3949f8e4ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:47.599646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e49763-23d2-47b3-841c-fd3949f8e4ee None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:47.614915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-b7e49763-23d2-47b3-841c-fd3949f8e4ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:47.614915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:47.614915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:47.614915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:47.614915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:47.614915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:47.614915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:47.614915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:47.614915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:47.614915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:47.614915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:47.614915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:47.614915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:47.614915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:47.614915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:47.615819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:47.615819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:47.615819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:47.615819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:47.615819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:47.615819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:47.615819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:47.615819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:47.615819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:47.615819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:47.615819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:47.615819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:47.615819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:47.615819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:47.615819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:47.615819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:47.615819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:47.617801 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:47.617801 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:47.617801 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:47.617801 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:47.617801 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 440/895] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:47.623660 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e49763-23d2-47b3-841c-fd3949f8e4ee None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:47.686034 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:47.715731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c3bfd-882e-442b-addd-ad2b915e00c3 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:47.752611 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e49763-23d2-47b3-841c-fd3949f8e4ee None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:47.760833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba97431-813f-4b9d-bfe2-91c4a77b1ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:47.761745 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba97431-813f-4b9d-bfe2-91c4a77b1ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:47.762224 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba97431-813f-4b9d-bfe2-91c4a77b1ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:47.762697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba97431-813f-4b9d-bfe2-91c4a77b1ca6 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:47.763585 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba97431-813f-4b9d-bfe2-91c4a77b1ca6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:47.764102 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba97431-813f-4b9d-bfe2-91c4a77b1ca6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:47.765732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba97431-813f-4b9d-bfe2-91c4a77b1ca6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:47.771397 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 456/896] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Aug 6 12:18:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:47.780348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba97431-813f-4b9d-bfe2-91c4a77b1ca6 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:47.787635 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:47.796591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac918e-6b50-443b-a886-3ab45fd94ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:47.797629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac918e-6b50-443b-a886-3ab45fd94ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:47.797915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac918e-6b50-443b-a886-3ab45fd94ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:47.798187 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac918e-6b50-443b-a886-3ab45fd94ea8 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:47.798888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac918e-6b50-443b-a886-3ab45fd94ea8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:47.799189 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac918e-6b50-443b-a886-3ab45fd94ea8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:47.800642 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac918e-6b50-443b-a886-3ab45fd94ea8 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:47.818194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 441/897] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:47.823861 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac918e-6b50-443b-a886-3ab45fd94ea8 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:47.890986 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba97431-813f-4b9d-bfe2-91c4a77b1ca6 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:47.930749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac918e-6b50-443b-a886-3ab45fd94ea8 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:47.931754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ce764-2ede-4acc-9137-88e69e8c3ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:47.932911 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ce764-2ede-4acc-9137-88e69e8c3ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:47.934147 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:47.935335 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ce764-2ede-4acc-9137-88e69e8c3ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:47.935842 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ce764-2ede-4acc-9137-88e69e8c3ce7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:47.936876 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129ce764-2ede-4acc-9137-88e69e8c3ce7 None admin] RBAC: Adding query filter params (interface=public, service_id=0050b6979ebe47cdbef64464e091157b, region_id=RegionOne) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:47.937400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129ce764-2ede-4acc-9137-88e69e8c3ce7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:47.939096 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129ce764-2ede-4acc-9137-88e69e8c3ce7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:47.948419 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 457/898] 10.222.0.20 () {56 vars in 1251 bytes} [Sat Aug 6 12:18:47 2022] GET /identity/v3/endpoints?service_id=0050b6979ebe47cdbef64464e091157b&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:47.953772 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ce764-2ede-4acc-9137-88e69e8c3ce7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:47.980791 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b224c-1c54-45ce-8d1b-f45fdbc4891a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:47.981429 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b224c-1c54-45ce-8d1b-f45fdbc4891a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:47.981761 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b224c-1c54-45ce-8d1b-f45fdbc4891a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:47.982036 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b224c-1c54-45ce-8d1b-f45fdbc4891a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:47.996172 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b224c-1c54-45ce-8d1b-f45fdbc4891a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:47.999324 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b224c-1c54-45ce-8d1b-f45fdbc4891a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:48.015696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-dc5b224c-1c54-45ce-8d1b-f45fdbc4891a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:48.015696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:48.015696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:48.015696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:48.015696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:48.015696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:48.015696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:48.015696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:48.015696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:48.015696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:48.015696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:48.015696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:48.015696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:48.015696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:48.015696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:48.016599 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:48.016599 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:48.016599 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:48.016599 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:48.016599 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:48.016599 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:48.016599 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:48.016599 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:48.016599 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:48.016599 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:48.016599 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:48.016599 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:48.016599 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:48.016599 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:48.016599 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:48.016599 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:48.016599 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:48.017488 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:48.017488 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:48.017488 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:48.017488 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:48.017488 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:48.017488 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:48.017488 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:48.017488 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:48.020254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 442/899] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:48.026040 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b224c-1c54-45ce-8d1b-f45fdbc4891a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:48.052361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:48.091290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ce764-2ede-4acc-9137-88e69e8c3ce7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:48.135349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8406e3cd-6b7b-4122-b087-887e5044a31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:48.136155 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406e3cd-6b7b-4122-b087-887e5044a31e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:48.136828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406e3cd-6b7b-4122-b087-887e5044a31e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:48.137208 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406e3cd-6b7b-4122-b087-887e5044a31e None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:48.137986 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8406e3cd-6b7b-4122-b087-887e5044a31e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:48.139922 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8406e3cd-6b7b-4122-b087-887e5044a31e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:48.156368 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 458/900] 10.222.0.20 () {56 vars in 1087 bytes} [Sat Aug 6 12:18:47 2022] GET /identity/v3/services => generated 1268 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:18:48.166434 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b224c-1c54-45ce-8d1b-f45fdbc4891a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:48.203737 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0301461-705a-4e75-9010-e3152dfdaf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:48.204616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0301461-705a-4e75-9010-e3152dfdaf9d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:48.204993 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0301461-705a-4e75-9010-e3152dfdaf9d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:48.205361 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0301461-705a-4e75-9010-e3152dfdaf9d None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:48.206229 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0301461-705a-4e75-9010-e3152dfdaf9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:48.206633 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0301461-705a-4e75-9010-e3152dfdaf9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:48.208211 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0301461-705a-4e75-9010-e3152dfdaf9d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:48.222711 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 443/901] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:48.229181 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8406e3cd-6b7b-4122-b087-887e5044a31e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:48.343845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8406e3cd-6b7b-4122-b087-887e5044a31e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:48.380151 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4df500-9b8a-49a4-8264-d7e4328de5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:48.380805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4df500-9b8a-49a4-8264-d7e4328de5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:48.381083 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4df500-9b8a-49a4-8264-d7e4328de5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:48.381416 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4df500-9b8a-49a4-8264-d7e4328de5e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:48.382204 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4df500-9b8a-49a4-8264-d7e4328de5e9 None admin] RBAC: Adding query filter params (role.id=ed864363c041447ab55d1795a8c7092b, scope.project.id=13fba33c46cf40bbb15a08c690b4c98c, user.id=c810b52785b94c0cab4fd9f0d6e65825) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:48.382589 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4df500-9b8a-49a4-8264-d7e4328de5e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:48.384045 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4df500-9b8a-49a4-8264-d7e4328de5e9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:48.387094 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:48.394162 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 459/902] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:18:48 2022] GET /identity/v3/role_assignments?user.id=c810b52785b94c0cab4fd9f0d6e65825&scope.project.id=13fba33c46cf40bbb15a08c690b4c98c&role.id=ed864363c041447ab55d1795a8c7092b => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:48.544798 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de434ab-af13-44aa-88c6-48e5de1e3e3b None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:48.545163 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de434ab-af13-44aa-88c6-48e5de1e3e3b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:48.545374 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de434ab-af13-44aa-88c6-48e5de1e3e3b None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:48.546948 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 444/903] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:48.552197 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd59c8-bc00-4050-909f-7afe871ffccb None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:48.552509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd59c8-bc00-4050-909f-7afe871ffccb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:48.552784 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd59c8-bc00-4050-909f-7afe871ffccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:48.572102 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1cbd59c8-bc00-4050-909f-7afe871ffccb None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:48.573305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1cbd59c8-bc00-4050-909f-7afe871ffccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:48.642719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 460/904] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:18:48.664450 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de434ab-af13-44aa-88c6-48e5de1e3e3b None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:48.691903 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:48.775449 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de434ab-af13-44aa-88c6-48e5de1e3e3b None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:48.811255 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363a51f-4061-456f-abf0-3bf5c986882e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:48.811903 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363a51f-4061-456f-abf0-3bf5c986882e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:48.812164 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363a51f-4061-456f-abf0-3bf5c986882e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:48.812475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363a51f-4061-456f-abf0-3bf5c986882e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:48.813115 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363a51f-4061-456f-abf0-3bf5c986882e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:48.814541 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363a51f-4061-456f-abf0-3bf5c986882e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:48.824483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-b363a51f-4061-456f-abf0-3bf5c986882e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 12:18:48.824483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:48.824483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:18:48.824483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:18:48.824483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:18:48.824483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:18:48.824483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:18:48.824483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:48.824483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 12:18:48.824483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:48.824483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:18:48.824483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:48.824483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:48.824483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:48.824483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:48.824483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:48.825446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:48.825446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:48.825446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:48.825446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:48.825446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:48.825446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:48.825446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:48.825446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:18:48.825446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:18:48.825446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:18:48.825446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:18:48.825446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:48.825446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:48.825446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:48.825446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:48.825446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:48.826298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:48.826298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:48.826298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:48.826298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:48.826298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:48.826298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:48.826298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:48.826298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:48.826298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:48.826298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:48.826298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:48.826298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:18:48.826298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:48.826298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 12:18:48.826298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:48.826298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 445/905] 10.222.0.20 () {56 vars in 1132 bytes} [Sat Aug 6 12:18:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:48.831610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd59c8-bc00-4050-909f-7afe871ffccb None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:48.938797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd59c8-bc00-4050-909f-7afe871ffccb None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:48.956711 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30f30c-0952-4fc4-bd8b-acffa9aa6652 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:48.957018 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30f30c-0952-4fc4-bd8b-acffa9aa6652 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:48.957275 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30f30c-0952-4fc4-bd8b-acffa9aa6652 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:48.958753 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 446/906] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:48.966219 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c1c82-7e42-466f-b50a-3b1770d81ad7 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:48.966510 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c1c82-7e42-466f-b50a-3b1770d81ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:48.966784 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c1c82-7e42-466f-b50a-3b1770d81ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:48.972995 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:48.983988 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8ca2f-47d0-4919-8405-9c49e959c33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:48.984648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8ca2f-47d0-4919-8405-9c49e959c33b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:48.984928 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8ca2f-47d0-4919-8405-9c49e959c33b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:48.985205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8ca2f-47d0-4919-8405-9c49e959c33b None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:48.985893 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f8ca2f-47d0-4919-8405-9c49e959c33b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:48.986196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f8ca2f-47d0-4919-8405-9c49e959c33b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:48.987230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-141c1c82-7e42-466f-b50a-3b1770d81ad7 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:48.987649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f8ca2f-47d0-4919-8405-9c49e959c33b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:48.988941 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-141c1c82-7e42-466f-b50a-3b1770d81ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:48.994934 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 461/907] 10.222.0.20 () {56 vars in 1126 bytes} [Sat Aug 6 12:18:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:48.999996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8ca2f-47d0-4919-8405-9c49e959c33b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:49.089373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 447/908] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 12:18:49.110844 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c1c82-7e42-466f-b50a-3b1770d81ad7 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:49.115107 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8ca2f-47d0-4919-8405-9c49e959c33b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:49.134672 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:49.162427 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d446bdf-fe2a-48a3-a8b0-1ec96aec053b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:49.163549 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d446bdf-fe2a-48a3-a8b0-1ec96aec053b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:49.164027 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d446bdf-fe2a-48a3-a8b0-1ec96aec053b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:49.165023 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d446bdf-fe2a-48a3-a8b0-1ec96aec053b None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:49.165885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d446bdf-fe2a-48a3-a8b0-1ec96aec053b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:49.166359 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d446bdf-fe2a-48a3-a8b0-1ec96aec053b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:49.169157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d446bdf-fe2a-48a3-a8b0-1ec96aec053b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:49.175392 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 462/909] 10.222.0.20 () {56 vars in 1126 bytes} [Sat Aug 6 12:18:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:49.183503 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d446bdf-fe2a-48a3-a8b0-1ec96aec053b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:49.240560 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c1c82-7e42-466f-b50a-3b1770d81ad7 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:49.250225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:49.292165 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e6aa5-7087-4060-ba58-92e4f534fbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:49.293087 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e6aa5-7087-4060-ba58-92e4f534fbff None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:49.293368 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e6aa5-7087-4060-ba58-92e4f534fbff None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:49.293641 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e6aa5-7087-4060-ba58-92e4f534fbff None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:49.294616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e6aa5-7087-4060-ba58-92e4f534fbff None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:49.296706 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e6aa5-7087-4060-ba58-92e4f534fbff None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:49.298010 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d446bdf-fe2a-48a3-a8b0-1ec96aec053b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:49.317477 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 448/910] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Aug 6 12:18:49 2022] POST /identity/v3/services => generated 241 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:18:49.346188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c527f3-2e58-43c4-a90b-1e36a5045a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:49.346822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c527f3-2e58-43c4-a90b-1e36a5045a04 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:49.347098 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c527f3-2e58-43c4-a90b-1e36a5045a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:49.347366 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c527f3-2e58-43c4-a90b-1e36a5045a04 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:49.348133 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c527f3-2e58-43c4-a90b-1e36a5045a04 None admin] RBAC: Adding query filter params (interface=public, service_id=ab5dd813eec84190b7879311c5650ea3, region_id=RegionOne) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:49.348474 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c527f3-2e58-43c4-a90b-1e36a5045a04 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:49.349860 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c527f3-2e58-43c4-a90b-1e36a5045a04 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:49.357298 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 463/911] 10.222.0.20 () {56 vars in 1251 bytes} [Sat Aug 6 12:18:49 2022] GET /identity/v3/endpoints?service_id=ab5dd813eec84190b7879311c5650ea3&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:49.362588 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e6aa5-7087-4060-ba58-92e4f534fbff None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:49.455265 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:49.479766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e6aa5-7087-4060-ba58-92e4f534fbff None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:49.515706 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9feafa-62bd-4086-acd0-af7078ef6e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:49.516257 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9feafa-62bd-4086-acd0-af7078ef6e00 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:49.516568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9feafa-62bd-4086-acd0-af7078ef6e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:49.516838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9feafa-62bd-4086-acd0-af7078ef6e00 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:49.517485 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9feafa-62bd-4086-acd0-af7078ef6e00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:49.518845 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9feafa-62bd-4086-acd0-af7078ef6e00 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:49.525321 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 449/912] 10.222.0.20 () {56 vars in 1087 bytes} [Sat Aug 6 12:18:49 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:18:50.316250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce965801-3bfe-45bb-91ba-f3116a50dfbd None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:50.317323 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce965801-3bfe-45bb-91ba-f3116a50dfbd None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:50.317811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce965801-3bfe-45bb-91ba-f3116a50dfbd None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:50.319645 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 464/913] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:50.325578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe0c78-21be-4800-be8e-5d34f037c6a1 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:50.326402 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe0c78-21be-4800-be8e-5d34f037c6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:50.327016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe0c78-21be-4800-be8e-5d34f037c6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:50.350211 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-2abe0c78-21be-4800-be8e-5d34f037c6a1 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:50.351768 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2abe0c78-21be-4800-be8e-5d34f037c6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:50.419049 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 450/914] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:18:50.442011 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce965801-3bfe-45bb-91ba-f3116a50dfbd None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:50.498758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa5707-c60d-444e-864f-2c0c58af53c1 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:50.499073 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa5707-c60d-444e-864f-2c0c58af53c1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:50.499323 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa5707-c60d-444e-864f-2c0c58af53c1 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:50.501026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 451/915] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:50.506024 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfa511-50da-4694-835b-280878882332 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:50.506311 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfa511-50da-4694-835b-280878882332 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:50.506840 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfa511-50da-4694-835b-280878882332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:50.526496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-52dfa511-50da-4694-835b-280878882332 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:50.527815 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-52dfa511-50da-4694-835b-280878882332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:50.554195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce965801-3bfe-45bb-91ba-f3116a50dfbd None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:50.573037 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:50.592006 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a38a1e-831b-4c1b-8937-d006d80a7942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:50.593279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a38a1e-831b-4c1b-8937-d006d80a7942 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:50.593999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a38a1e-831b-4c1b-8937-d006d80a7942 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:50.594504 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a38a1e-831b-4c1b-8937-d006d80a7942 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:50.595733 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a38a1e-831b-4c1b-8937-d006d80a7942 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:50.597602 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a38a1e-831b-4c1b-8937-d006d80a7942 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:50.604559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 452/916] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 12:18:50.607347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-98a38a1e-831b-4c1b-8937-d006d80a7942 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 12:18:50.607347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:50.607347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:18:50.607347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:18:50.607347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:18:50.607347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:18:50.607347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:18:50.607347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:50.607347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 12:18:50.607347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:50.607347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:18:50.607347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:50.607347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:50.607347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:50.607347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:50.607347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:50.607347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:50.608332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:50.608332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:50.608332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:50.608332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:50.608332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:50.608332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:50.608332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:18:50.608332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:18:50.608332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:18:50.608332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:18:50.608332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:50.608332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:50.608332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:50.608332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:50.608332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:50.608332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:50.609186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:50.609186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:50.609186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:50.609186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:50.609186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:50.609186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:50.609186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:50.609186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:50.609186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:50.609186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:50.609186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:18:50.609186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:50.609186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 06 12:18:50.609186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:50.616395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 465/917] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Aug 6 12:18:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:50.618810 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfa511-50da-4694-835b-280878882332 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:50.623576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:50.624699 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a38a1e-831b-4c1b-8937-d006d80a7942 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:50.732671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfa511-50da-4694-835b-280878882332 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:50.755122 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a38a1e-831b-4c1b-8937-d006d80a7942 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:50.769291 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae59b17-f894-42c9-bba3-a91267c93dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:50.769937 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae59b17-f894-42c9-bba3-a91267c93dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:50.770222 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae59b17-f894-42c9-bba3-a91267c93dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:50.770511 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae59b17-f894-42c9-bba3-a91267c93dc5 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:50.771186 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae59b17-f894-42c9-bba3-a91267c93dc5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:50.771543 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae59b17-f894-42c9-bba3-a91267c93dc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:50.773604 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae59b17-f894-42c9-bba3-a91267c93dc5 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:50.779769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 453/918] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Aug 6 12:18:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:50.786945 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae59b17-f894-42c9-bba3-a91267c93dc5 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:50.801194 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571d2b9-cb43-4cbe-9edb-859319ab9735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:50.802022 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571d2b9-cb43-4cbe-9edb-859319ab9735 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:50.802514 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571d2b9-cb43-4cbe-9edb-859319ab9735 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:50.802763 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571d2b9-cb43-4cbe-9edb-859319ab9735 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:50.813803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2571d2b9-cb43-4cbe-9edb-859319ab9735 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:50.819613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-2571d2b9-cb43-4cbe-9edb-859319ab9735 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 12:18:50.819613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:50.819613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:50.819613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:50.819613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:50.819613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:50.819613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:50.819613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:50.819613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:50.819613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:50.819613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:50.819613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:50.819613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:50.819613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:50.819613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:50.819613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:50.820809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:50.820809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:50.820809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:50.820809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:50.820809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:50.820809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:50.820809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:50.820809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:50.820809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:50.820809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:50.820809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:50.820809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:50.820809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:50.820809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:50.820809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:50.820809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:50.821840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:50.821840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:50.821840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:50.821840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:50.821840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:50.821840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:50.821840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:50.821840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:50.821840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:50.821840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:50.821840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 12:18:50.821840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:50.828528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 466/919] 10.222.0.20 () {56 vars in 1120 bytes} [Sat Aug 6 12:18:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:50.829529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571d2b9-cb43-4cbe-9edb-859319ab9735 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:50.875076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:50.876684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:50.916876 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae59b17-f894-42c9-bba3-a91267c93dc5 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:50.941616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571d2b9-cb43-4cbe-9edb-859319ab9735 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:50.959684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1fcab-b2a7-4b09-8511-4c3534a52ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:50.960831 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1fcab-b2a7-4b09-8511-4c3534a52ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:50.960831 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1fcab-b2a7-4b09-8511-4c3534a52ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:50.961326 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1fcab-b2a7-4b09-8511-4c3534a52ad6 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:50.962223 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1fcab-b2a7-4b09-8511-4c3534a52ad6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:50.962549 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1fcab-b2a7-4b09-8511-4c3534a52ad6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:50.964010 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1fcab-b2a7-4b09-8511-4c3534a52ad6 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:50.973751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 454/920] 10.222.0.20 () {56 vars in 1114 bytes} [Sat Aug 6 12:18:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:50.975266 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1fcab-b2a7-4b09-8511-4c3534a52ad6 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:50.988591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f7d9d-b88c-40c5-85f7-018128b66a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:50.989459 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f7d9d-b88c-40c5-85f7-018128b66a41 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:50.989596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f7d9d-b88c-40c5-85f7-018128b66a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:50.989918 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f7d9d-b88c-40c5-85f7-018128b66a41 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:50.990709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f7d9d-b88c-40c5-85f7-018128b66a41 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:50.991381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f7d9d-b88c-40c5-85f7-018128b66a41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:50.993479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f7d9d-b88c-40c5-85f7-018128b66a41 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:51.007956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 467/921] 10.222.0.20 () {56 vars in 1115 bytes} [Sat Aug 6 12:18:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:51.009015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f7d9d-b88c-40c5-85f7-018128b66a41 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:51.109269 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1fcab-b2a7-4b09-8511-4c3534a52ad6 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:51.118981 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f7d9d-b88c-40c5-85f7-018128b66a41 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:51.122641 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:51.151309 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:51.163023 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243f078-7a17-4995-a1da-52d41c45b0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:51.163643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243f078-7a17-4995-a1da-52d41c45b0c3 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:51.163962 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243f078-7a17-4995-a1da-52d41c45b0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:51.164235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243f078-7a17-4995-a1da-52d41c45b0c3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:51.165449 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243f078-7a17-4995-a1da-52d41c45b0c3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:51.166812 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243f078-7a17-4995-a1da-52d41c45b0c3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:51.170212 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236bcb17-b8a5-4553-af1a-5a3717d4d174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:51.171170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bcb17-b8a5-4553-af1a-5a3717d4d174 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:51.171924 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bcb17-b8a5-4553-af1a-5a3717d4d174 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:51.172885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bcb17-b8a5-4553-af1a-5a3717d4d174 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:51.187861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236bcb17-b8a5-4553-af1a-5a3717d4d174 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:51.189733 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236bcb17-b8a5-4553-af1a-5a3717d4d174 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:51.200331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 455/922] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:18:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 12:18:51.215873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-236bcb17-b8a5-4553-af1a-5a3717d4d174 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:51.215873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:51.215873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:51.215873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:51.215873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:51.215873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:51.215873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:51.215873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:51.215873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:51.215873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:51.215873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:51.215873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:51.215873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:51.215873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:51.215873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:51.216983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:51.216983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:51.216983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:51.216983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:51.216983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:51.216983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:51.216983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:51.216983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:51.216983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:51.216983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:51.216983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:51.216983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:51.216983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:51.216983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:51.216983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:51.216983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:51.216983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:51.217867 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:51.217867 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:51.217867 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:51.217867 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:51.219945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 468/923] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:51.225903 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243f078-7a17-4995-a1da-52d41c45b0c3 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:51.346579 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243f078-7a17-4995-a1da-52d41c45b0c3 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:51.393126 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07055c-c6d9-4c9d-8ef2-6cdfabdfe21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:51.393933 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07055c-c6d9-4c9d-8ef2-6cdfabdfe21a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:51.394358 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07055c-c6d9-4c9d-8ef2-6cdfabdfe21a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:51.394776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07055c-c6d9-4c9d-8ef2-6cdfabdfe21a None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:51.395852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07055c-c6d9-4c9d-8ef2-6cdfabdfe21a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:51.396340 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07055c-c6d9-4c9d-8ef2-6cdfabdfe21a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:51.399034 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07055c-c6d9-4c9d-8ef2-6cdfabdfe21a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:51.410782 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376bb6a3-223f-4a70-b21c-cfe3c88be6ff None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:51.411272 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376bb6a3-223f-4a70-b21c-cfe3c88be6ff None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:51.411349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376bb6a3-223f-4a70-b21c-cfe3c88be6ff None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:51.413012 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 469/924] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:18:51.417524 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 456/925] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:51.419414 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a78ee2-7889-4ac8-9c27-5a8dbded7a0c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:51.419827 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a78ee2-7889-4ac8-9c27-5a8dbded7a0c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:51.420234 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a78ee2-7889-4ac8-9c27-5a8dbded7a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:51.422904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07055c-c6d9-4c9d-8ef2-6cdfabdfe21a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:51.440040 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-20a78ee2-7889-4ac8-9c27-5a8dbded7a0c None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:51.441974 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-20a78ee2-7889-4ac8-9c27-5a8dbded7a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:51.481180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:51.544830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 470/926] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:18:51.557473 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07055c-c6d9-4c9d-8ef2-6cdfabdfe21a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:51.562087 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a78ee2-7889-4ac8-9c27-5a8dbded7a0c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:51.598745 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10966198-f9b0-4035-8404-ec056b310cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:51.599371 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10966198-f9b0-4035-8404-ec056b310cba None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:51.599644 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10966198-f9b0-4035-8404-ec056b310cba None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:51.599911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10966198-f9b0-4035-8404-ec056b310cba None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:51.607753 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10966198-f9b0-4035-8404-ec056b310cba None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:51.609235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10966198-f9b0-4035-8404-ec056b310cba None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:51.621228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-10966198-f9b0-4035-8404-ec056b310cba None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 12:18:51.621228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:51.621228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:51.621228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:51.621228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:51.621228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:51.621228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:51.621228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:51.621228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:51.621228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:51.621228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:51.621228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:51.621228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:51.621228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:51.621228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:51.621228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:51.622284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:51.622284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:51.622284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:51.622284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:51.622284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:51.622284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:51.622284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:51.622284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:51.622284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:51.622284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:51.622284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:51.622284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:51.622284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:51.622284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:51.622284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:51.622284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:51.622284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:51.622284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:51.623243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:51.623243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:51.623243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:51.623243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:51.623243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:51.623243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:51.623243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:51.623243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:51.623243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 12:18:51.623243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:51.627338 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 457/927] 10.222.0.20 () {56 vars in 1117 bytes} [Sat Aug 6 12:18:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:51.628639 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947105f2-d037-430b-a1ce-d76a4fa46a4e None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:51.628923 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947105f2-d037-430b-a1ce-d76a4fa46a4e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:51.629204 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947105f2-d037-430b-a1ce-d76a4fa46a4e None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:51.630488 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 458/928] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:51.631171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947105f2-d037-430b-a1ce-d76a4fa46a4e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:51.657897 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:51.674734 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a78ee2-7889-4ac8-9c27-5a8dbded7a0c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:51.709462 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7d059-3e62-4451-beba-9ec766cf3807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:51.710106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7d059-3e62-4451-beba-9ec766cf3807 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:51.710387 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7d059-3e62-4451-beba-9ec766cf3807 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:51.711200 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7d059-3e62-4451-beba-9ec766cf3807 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:51.711828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a7d059-3e62-4451-beba-9ec766cf3807 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:51.713217 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a7d059-3e62-4451-beba-9ec766cf3807 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:51.722196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-e9a7d059-3e62-4451-beba-9ec766cf3807 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 12:18:51.722196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:51.722196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:18:51.722196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:18:51.722196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:18:51.722196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:18:51.722196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:18:51.722196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:51.722196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 12:18:51.722196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:51.722196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:18:51.722196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:51.722196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:51.722196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:51.722196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:51.722196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:51.722196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:51.723382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:51.723382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:51.723382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:51.723382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:51.723382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:51.723382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:51.723382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:18:51.723382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:18:51.723382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:18:51.723382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:18:51.723382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:51.723382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:51.723382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:51.723382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:51.723382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:51.723382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:51.724221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:51.724221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:51.724221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:51.724221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:51.724221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:51.724221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:51.724221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:51.724221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:51.724221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:51.724221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:51.724221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:18:51.724221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:51.724221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 12:18:51.724221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:51.724221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 471/929] 10.222.0.20 () {56 vars in 1126 bytes} [Sat Aug 6 12:18:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:51.725372 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ce92b-432b-436c-ac83-8a9da80e90ec None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:51.725372 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ce92b-432b-436c-ac83-8a9da80e90ec None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:51.725372 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ce92b-432b-436c-ac83-8a9da80e90ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:51.743482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947105f2-d037-430b-a1ce-d76a4fa46a4e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:51.745402 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:51.751258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ac1ce92b-432b-436c-ac83-8a9da80e90ec None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:51.752600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ac1ce92b-432b-436c-ac83-8a9da80e90ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:51.782757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e795a0-3039-4a5c-a37f-ff999e00c62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:51.783366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e795a0-3039-4a5c-a37f-ff999e00c62f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:51.783641 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e795a0-3039-4a5c-a37f-ff999e00c62f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:51.783912 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e795a0-3039-4a5c-a37f-ff999e00c62f None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:51.784734 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e795a0-3039-4a5c-a37f-ff999e00c62f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:51.785036 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e795a0-3039-4a5c-a37f-ff999e00c62f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:51.786447 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e795a0-3039-4a5c-a37f-ff999e00c62f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:51.806164 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 459/930] 10.222.0.20 () {56 vars in 1149 bytes} [Sat Aug 6 12:18:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:51.807364 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e795a0-3039-4a5c-a37f-ff999e00c62f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:51.834688 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 472/931] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:18:51.835897 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ce92b-432b-436c-ac83-8a9da80e90ec None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:51.922530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:51.923612 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e795a0-3039-4a5c-a37f-ff999e00c62f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:51.942054 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ce92b-432b-436c-ac83-8a9da80e90ec None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:51.965939 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295933fc-8e89-493d-9736-62b8bb4bb17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:51.966547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295933fc-8e89-493d-9736-62b8bb4bb17a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:51.966825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295933fc-8e89-493d-9736-62b8bb4bb17a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:51.967104 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295933fc-8e89-493d-9736-62b8bb4bb17a None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:51.967764 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295933fc-8e89-493d-9736-62b8bb4bb17a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:51.968064 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295933fc-8e89-493d-9736-62b8bb4bb17a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:51.969480 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295933fc-8e89-493d-9736-62b8bb4bb17a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:51.974327 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 460/932] 10.222.0.20 () {56 vars in 1122 bytes} [Sat Aug 6 12:18:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:51.975100 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295933fc-8e89-493d-9736-62b8bb4bb17a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:51.982129 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952c4bb-6947-4159-a5e9-931c043430e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:51.982785 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952c4bb-6947-4159-a5e9-931c043430e9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:51.983815 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952c4bb-6947-4159-a5e9-931c043430e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:51.983815 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952c4bb-6947-4159-a5e9-931c043430e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:51.997243 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2952c4bb-6947-4159-a5e9-931c043430e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:51.998933 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2952c4bb-6947-4159-a5e9-931c043430e9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:52.009839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:52.014841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-2952c4bb-6947-4159-a5e9-931c043430e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:52.014841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:52.014841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:52.014841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:52.014841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:52.014841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:52.014841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:52.014841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:52.014841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:52.014841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:52.014841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:52.014841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:52.014841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:52.014841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:52.014841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:52.015826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:52.015826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:52.015826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:52.015826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:52.015826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:52.015826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:52.015826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:52.015826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:52.015826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:52.015826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:52.015826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:52.015826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:52.015826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:52.015826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:52.015826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:52.015826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:52.015826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:52.016777 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:52.016777 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:52.016777 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:52.016777 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:52.016777 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 473/933] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:52.016777 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952c4bb-6947-4159-a5e9-931c043430e9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:52.091485 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295933fc-8e89-493d-9736-62b8bb4bb17a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:52.125324 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952c4bb-6947-4159-a5e9-931c043430e9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:52.130300 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e7a99-1393-43d8-9198-ce93c9e0a6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:52.130938 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e7a99-1393-43d8-9198-ce93c9e0a6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:52.131208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e7a99-1393-43d8-9198-ce93c9e0a6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:52.131473 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e7a99-1393-43d8-9198-ce93c9e0a6f2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:52.132095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e7a99-1393-43d8-9198-ce93c9e0a6f2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:52.133528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e7a99-1393-43d8-9198-ce93c9e0a6f2 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:52.142981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-ba3e7a99-1393-43d8-9198-ce93c9e0a6f2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 12:18:52.142981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:52.142981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:18:52.142981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:18:52.142981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:18:52.142981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:18:52.142981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:18:52.142981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:52.142981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 12:18:52.142981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:52.142981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:18:52.142981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:52.142981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:52.142981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:52.142981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:52.142981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:52.143931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:52.143931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:52.143931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:52.143931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:52.143931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:52.143931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:52.143931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:52.143931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:18:52.143931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:18:52.143931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:18:52.143931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:18:52.143931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:52.143931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:52.143931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:52.143931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:52.143931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:52.144807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:52.144807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:52.144807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:52.144807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:52.144807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:52.144807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:52.144807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:52.144807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:52.144807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:52.144807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:52.144807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:52.144807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:18:52.144807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:52.144807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 06 12:18:52.144807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:52.144807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 461/934] 10.222.0.20 () {56 vars in 1132 bytes} [Sat Aug 6 12:18:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:52.145634 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e7a99-1393-43d8-9198-ce93c9e0a6f2 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:52.167668 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:52.174342 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465b41e-83e1-44e6-b1ca-9e078aca9be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:52.174994 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465b41e-83e1-44e6-b1ca-9e078aca9be6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:52.175183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465b41e-83e1-44e6-b1ca-9e078aca9be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:52.175461 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465b41e-83e1-44e6-b1ca-9e078aca9be6 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:52.176143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465b41e-83e1-44e6-b1ca-9e078aca9be6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:52.176481 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465b41e-83e1-44e6-b1ca-9e078aca9be6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:52.177840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465b41e-83e1-44e6-b1ca-9e078aca9be6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:52.183701 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 474/935] 10.222.0.20 () {56 vars in 1122 bytes} [Sat Aug 6 12:18:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:52.183978 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465b41e-83e1-44e6-b1ca-9e078aca9be6 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:52.243137 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:52.256248 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e7a99-1393-43d8-9198-ce93c9e0a6f2 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:52.287668 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465b41e-83e1-44e6-b1ca-9e078aca9be6 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:52.292035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0db52-8302-4181-bf65-25c8edbb2bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:52.292679 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0db52-8302-4181-bf65-25c8edbb2bfb None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:52.292959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0db52-8302-4181-bf65-25c8edbb2bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:52.293225 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0db52-8302-4181-bf65-25c8edbb2bfb None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:52.293903 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0db52-8302-4181-bf65-25c8edbb2bfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:52.294201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0db52-8302-4181-bf65-25c8edbb2bfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:52.295546 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0db52-8302-4181-bf65-25c8edbb2bfb None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:52.308397 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 462/936] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:52.309012 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0db52-8302-4181-bf65-25c8edbb2bfb None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:52.329985 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313213ae-12f0-4299-ba50-2110e02c7922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:52.330612 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313213ae-12f0-4299-ba50-2110e02c7922 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:52.331012 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313213ae-12f0-4299-ba50-2110e02c7922 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:52.331293 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313213ae-12f0-4299-ba50-2110e02c7922 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:52.331973 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313213ae-12f0-4299-ba50-2110e02c7922 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:52.332279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313213ae-12f0-4299-ba50-2110e02c7922 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:52.333779 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313213ae-12f0-4299-ba50-2110e02c7922 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:52.343634 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 475/937] 10.222.0.20 () {56 vars in 1126 bytes} [Sat Aug 6 12:18:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:52.344338 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313213ae-12f0-4299-ba50-2110e02c7922 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:52.406788 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:52.416666 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0db52-8302-4181-bf65-25c8edbb2bfb None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:52.450447 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313213ae-12f0-4299-ba50-2110e02c7922 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:52.454234 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805477fa-6850-450f-abac-933608cc5e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:52.454844 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805477fa-6850-450f-abac-933608cc5e8d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:52.455124 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805477fa-6850-450f-abac-933608cc5e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:52.455396 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805477fa-6850-450f-abac-933608cc5e8d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:52.456150 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805477fa-6850-450f-abac-933608cc5e8d None admin] RBAC: Adding query filter params (interface=public, service_id=8de0bfb4caab448a8de667770f89657c, region_id=RegionOne) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:52.456498 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805477fa-6850-450f-abac-933608cc5e8d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:52.458002 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805477fa-6850-450f-abac-933608cc5e8d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:52.467747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 463/938] 10.222.0.20 () {56 vars in 1251 bytes} [Sat Aug 6 12:18:52 2022] GET /identity/v3/endpoints?service_id=8de0bfb4caab448a8de667770f89657c&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:52.467747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805477fa-6850-450f-abac-933608cc5e8d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:52.480647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:52.492719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d9e09-8991-4a53-bb19-c052af281f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:52.493361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d9e09-8991-4a53-bb19-c052af281f96 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:52.493667 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d9e09-8991-4a53-bb19-c052af281f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:52.493932 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d9e09-8991-4a53-bb19-c052af281f96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:52.500595 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d9e09-8991-4a53-bb19-c052af281f96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:52.502022 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d9e09-8991-4a53-bb19-c052af281f96 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:52.513713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-da4d9e09-8991-4a53-bb19-c052af281f96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:52.513713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:52.513713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:52.513713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:52.513713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:52.513713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:52.513713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:52.513713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:52.513713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:52.513713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:52.513713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:52.513713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:52.513713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:52.513713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:52.513713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:52.514598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:52.514598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:52.514598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:52.514598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:52.514598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:52.514598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:52.514598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:52.514598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:52.514598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:52.514598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:52.514598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:52.514598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:52.514598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:52.514598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:52.514598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:52.514598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:52.514598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:52.515482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:52.515482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:52.515482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:52.515482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:52.515482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:52.515482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:52.515482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:52.515482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:52.515482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 476/939] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:52.516253 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d9e09-8991-4a53-bb19-c052af281f96 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:52.579736 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805477fa-6850-450f-abac-933608cc5e8d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:52.617703 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa5d35-d9d5-433a-9288-3bf8c70119a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:52.618308 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa5d35-d9d5-433a-9288-3bf8c70119a7 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:52.618585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa5d35-d9d5-433a-9288-3bf8c70119a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:52.618856 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa5d35-d9d5-433a-9288-3bf8c70119a7 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:52.619516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa5d35-d9d5-433a-9288-3bf8c70119a7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:52.619824 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa5d35-d9d5-433a-9288-3bf8c70119a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:52.621238 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa5d35-d9d5-433a-9288-3bf8c70119a7 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:52.623682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d9e09-8991-4a53-bb19-c052af281f96 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:52.631185 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 464/940] 10.222.0.20 () {56 vars in 1126 bytes} [Sat Aug 6 12:18:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:52.631922 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa5d35-d9d5-433a-9288-3bf8c70119a7 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:52.647106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:52.665027 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382df5d-2bae-4a50-9545-625e629118ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:52.666548 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382df5d-2bae-4a50-9545-625e629118ff None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:52.666831 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382df5d-2bae-4a50-9545-625e629118ff None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:52.667103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382df5d-2bae-4a50-9545-625e629118ff None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:52.667748 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8382df5d-2bae-4a50-9545-625e629118ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:52.669193 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8382df5d-2bae-4a50-9545-625e629118ff None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:52.685906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 477/941] 10.222.0.20 () {56 vars in 1087 bytes} [Sat Aug 6 12:18:52 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:18:52.685906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382df5d-2bae-4a50-9545-625e629118ff None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:52.715562 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:52.749777 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa5d35-d9d5-433a-9288-3bf8c70119a7 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:52.786719 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105b5c8-891d-46d0-834a-cfaad6034252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:52.787370 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105b5c8-891d-46d0-834a-cfaad6034252 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:52.787646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105b5c8-891d-46d0-834a-cfaad6034252 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:52.787916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105b5c8-891d-46d0-834a-cfaad6034252 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:52.789291 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9105b5c8-891d-46d0-834a-cfaad6034252 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:52.789642 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9105b5c8-891d-46d0-834a-cfaad6034252 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:52.791197 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9105b5c8-891d-46d0-834a-cfaad6034252 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:52.805082 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 465/942] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:52.810073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382df5d-2bae-4a50-9545-625e629118ff None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:52.812682 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105b5c8-891d-46d0-834a-cfaad6034252 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:52.858184 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd26c26-7d98-4579-85f0-dcbbbb1e2038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:52.859647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd26c26-7d98-4579-85f0-dcbbbb1e2038 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:52.860132 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd26c26-7d98-4579-85f0-dcbbbb1e2038 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:52.860663 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd26c26-7d98-4579-85f0-dcbbbb1e2038 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:52.861936 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd26c26-7d98-4579-85f0-dcbbbb1e2038 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:52.864130 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd26c26-7d98-4579-85f0-dcbbbb1e2038 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:52.891878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 478/943] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:18:52 2022] POST /identity/v3/endpoints => generated 347 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:18:52.930931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105b5c8-891d-46d0-834a-cfaad6034252 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:52.974725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:52.981260 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22ec9c-a4b7-4c44-8f66-fc6475f39f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:52.982292 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22ec9c-a4b7-4c44-8f66-fc6475f39f74 None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:52.982772 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22ec9c-a4b7-4c44-8f66-fc6475f39f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:52.983247 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22ec9c-a4b7-4c44-8f66-fc6475f39f74 None admin] PATH_INFO: `/v3/projects/13fba33c46cf40bbb15a08c690b4c98c/users/c810b52785b94c0cab4fd9f0d6e65825/roles/ed864363c041447ab55d1795a8c7092b` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:52.985875 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22ec9c-a4b7-4c44-8f66-fc6475f39f74 None admin] RBAC: Authorizing `identity:create_grant(project_id=13fba33c46cf40bbb15a08c690b4c98c, user_id=c810b52785b94c0cab4fd9f0d6e65825, role_id=ed864363c041447ab55d1795a8c7092b)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:52.987631 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22ec9c-a4b7-4c44-8f66-fc6475f39f74 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:53.005281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 466/944] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:18:52 2022] PUT /identity/v3/projects/13fba33c46cf40bbb15a08c690b4c98c/users/c810b52785b94c0cab4fd9f0d6e65825/roles/ed864363c041447ab55d1795a8c7092b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:18:53.349223 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe185e95-f891-469f-a9e0-4809e8109d2f None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:53.349625 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe185e95-f891-469f-a9e0-4809e8109d2f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:53.349828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe185e95-f891-469f-a9e0-4809e8109d2f None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:53.351917 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 479/945] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:53.357274 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6cd1a-b8f5-4cc1-af5e-080317e67d2d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:53.357566 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6cd1a-b8f5-4cc1-af5e-080317e67d2d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:53.357849 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6cd1a-b8f5-4cc1-af5e-080317e67d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:53.376802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c3e6cd1a-b8f5-4cc1-af5e-080317e67d2d None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:53.378000 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c3e6cd1a-b8f5-4cc1-af5e-080317e67d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:53.456480 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 467/946] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:18:53.490692 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe185e95-f891-469f-a9e0-4809e8109d2f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:53.539893 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:53.606289 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe185e95-f891-469f-a9e0-4809e8109d2f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:53.642909 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df18d8d-6843-4ffd-a27d-32504c1cf12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:53.643529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df18d8d-6843-4ffd-a27d-32504c1cf12d None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:53.643808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df18d8d-6843-4ffd-a27d-32504c1cf12d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:53.644080 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df18d8d-6843-4ffd-a27d-32504c1cf12d None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:53.644857 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df18d8d-6843-4ffd-a27d-32504c1cf12d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:53.646361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df18d8d-6843-4ffd-a27d-32504c1cf12d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:53.649059 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: WARNING keystone.server.flask.common [None req-1df18d8d-6843-4ffd-a27d-32504c1cf12d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 06 12:18:53.667704 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.sql.core [None req-1df18d8d-6843-4ffd-a27d-32504c1cf12d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 06 12:18:53.667704 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 06 12:18:53.667704 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [parameters: {'id': 'c6ffd75cfe414f56bfe227ea2c36303d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 06 12:18:53.667704 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80144) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 06 12:18:53.675046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-1df18d8d-6843-4ffd-a27d-32504c1cf12d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c6ffd75cfe414f56bfe227ea2c36303d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c6ffd75cfe414f56bfe227ea2c36303d. Aug 06 12:18:53.675046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:53.675046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 12:18:53.675046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 12:18:53.675046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 12:18:53.675046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 12:18:53.675046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 12:18:53.675046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 12:18:53.675046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 12:18:53.675046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application conn.query(q) Aug 06 12:18:53.675046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 12:18:53.675046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 12:18:53.675046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 12:18:53.676843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application result.read() Aug 06 12:18:53.676843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 12:18:53.676843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 12:18:53.676843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 12:18:53.676843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 12:18:53.676843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 12:18:53.676843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 12:18:53.676843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 12:18:53.676843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 12:18:53.676843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 06 12:18:53.676843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:53.676843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 06 12:18:53.676843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:53.676843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:53.676843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 06 12:18:53.676843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 06 12:18:53.676843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 06 12:18:53.676843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 06 12:18:53.679049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 12:18:53.679049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application next(self.gen) Aug 06 12:18:53.679049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 06 12:18:53.679049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application yield resource Aug 06 12:18:53.679049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 12:18:53.679049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application next(self.gen) Aug 06 12:18:53.679049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 06 12:18:53.679049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.session.rollback() Aug 06 12:18:53.679049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 06 12:18:53.679049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.force_reraise() Aug 06 12:18:53.679049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 06 12:18:53.679049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise self.value Aug 06 12:18:53.679049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 06 12:18:53.679049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 06 12:18:53.679049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 06 12:18:53.679049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application session.commit() Aug 06 12:18:53.679049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 06 12:18:53.681464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 06 12:18:53.681464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 06 12:18:53.681464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._prepare_impl() Aug 06 12:18:53.681464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 06 12:18:53.681464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.session.flush() Aug 06 12:18:53.681464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 06 12:18:53.681464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._flush(objects) Aug 06 12:18:53.681464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 06 12:18:53.681464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 06 12:18:53.681464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 06 12:18:53.681464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application compat.raise_( Aug 06 12:18:53.681464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 12:18:53.681464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception Aug 06 12:18:53.681464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 06 12:18:53.681464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application flush_context.execute() Aug 06 12:18:53.681464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 06 12:18:53.681464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rec.execute(self) Aug 06 12:18:53.682615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 06 12:18:53.682615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 06 12:18:53.682615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 06 12:18:53.682615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 06 12:18:53.682615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 06 12:18:53.682615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 06 12:18:53.682615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 06 12:18:53.682615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 06 12:18:53.682615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 06 12:18:53.682615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 06 12:18:53.682615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 06 12:18:53.682615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 06 12:18:53.682615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 06 12:18:53.682615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 06 12:18:53.682615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 06 12:18:53.683629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 06 12:18:53.683629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 12:18:53.683629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception Aug 06 12:18:53.683629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 12:18:53.683629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 12:18:53.683629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 12:18:53.683629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 12:18:53.683629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 12:18:53.683629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 12:18:53.683629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 12:18:53.683629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application conn.query(q) Aug 06 12:18:53.683629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 12:18:53.683629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 12:18:53.683629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 12:18:53.683629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application result.read() Aug 06 12:18:53.683629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 12:18:53.683629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 12:18:53.684747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 12:18:53.684747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 12:18:53.684747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 12:18:53.684747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 12:18:53.684747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 12:18:53.684747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 12:18:53.684747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 06 12:18:53.684747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 06 12:18:53.684747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [parameters: {'id': 'c6ffd75cfe414f56bfe227ea2c36303d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 06 12:18:53.684747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 06 12:18:53.684747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:53.684747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:18:53.684747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:53.684747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:53.684747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 06 12:18:53.685753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 06 12:18:53.685753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 06 12:18:53.685753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 06 12:18:53.685753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 06 12:18:53.685753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:53.685753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:18:53.685753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:53.685753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:53.685753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:53.685753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:53.685753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:53.685753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:53.685753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:53.685753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:53.685753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:53.685753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:53.685753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:53.686907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:53.686907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 06 12:18:53.686907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 06 12:18:53.686907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:53.686907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:53.686907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 06 12:18:53.686907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 06 12:18:53.686907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c6ffd75cfe414f56bfe227ea2c36303d. Aug 06 12:18:53.686907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:53.686907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 480/947] 10.222.0.20 () {60 vars in 1140 bytes} [Sat Aug 6 12:18:53 2022] POST /identity/v3/projects => generated 262 bytes in 187 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:53.688822 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6cd1a-b8f5-4cc1-af5e-080317e67d2d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:53.804661 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6cd1a-b8f5-4cc1-af5e-080317e67d2d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:53.828489 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:53.841560 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7328c050-b9a2-431f-96e7-f317feff13aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:53.843192 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328c050-b9a2-431f-96e7-f317feff13aa None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:53.843627 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328c050-b9a2-431f-96e7-f317feff13aa None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:53.844468 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328c050-b9a2-431f-96e7-f317feff13aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:53.852363 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7328c050-b9a2-431f-96e7-f317feff13aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:53.854020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7328c050-b9a2-431f-96e7-f317feff13aa None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:53.865828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-7328c050-b9a2-431f-96e7-f317feff13aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:53.865828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:53.865828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:53.865828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:53.865828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:53.865828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:53.865828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:53.865828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:53.865828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:53.865828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:53.865828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:53.865828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:53.865828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:53.865828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:53.865828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:53.867643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:53.867643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:53.867643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:53.867643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:53.867643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:53.867643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:53.867643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:53.867643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:53.867643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:53.867643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:53.867643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:53.867643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:53.867643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:53.867643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:53.867643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:53.867643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:53.867643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:53.868775 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:53.868775 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:53.868775 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:53.868775 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:53.868775 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:53.868775 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:53.868775 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:53.868775 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:53.870884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 468/948] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:53.876657 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df18d8d-6843-4ffd-a27d-32504c1cf12d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:53.979466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:53.986006 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df18d8d-6843-4ffd-a27d-32504c1cf12d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:54.030071 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db1c79-309d-45e5-8ef9-b065af50ab0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:54.030071 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db1c79-309d-45e5-8ef9-b065af50ab0e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:54.030071 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db1c79-309d-45e5-8ef9-b065af50ab0e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:54.030071 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db1c79-309d-45e5-8ef9-b065af50ab0e None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:54.030542 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db1c79-309d-45e5-8ef9-b065af50ab0e None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:54.030542 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db1c79-309d-45e5-8ef9-b065af50ab0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:54.030542 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db1c79-309d-45e5-8ef9-b065af50ab0e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:54.050827 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 481/949] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Aug 6 12:18:53 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:54.918585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc663c-197f-4d6f-9558-29bac9a8d61a None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:54.919233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc663c-197f-4d6f-9558-29bac9a8d61a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:54.919233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc663c-197f-4d6f-9558-29bac9a8d61a None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:54.920735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 469/950] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:54.926010 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e8299-0bf7-4cf7-9ebc-5e378d14d2fb None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:54.926427 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e8299-0bf7-4cf7-9ebc-5e378d14d2fb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:54.926602 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e8299-0bf7-4cf7-9ebc-5e378d14d2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:54.945619 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d29e8299-0bf7-4cf7-9ebc-5e378d14d2fb None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:54.947106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d29e8299-0bf7-4cf7-9ebc-5e378d14d2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:55.009329 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc455b-1675-4a3c-869d-bfaab47db282 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:55.009613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc455b-1675-4a3c-869d-bfaab47db282 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:55.009887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc455b-1675-4a3c-869d-bfaab47db282 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:55.011454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 470/951] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:55.016523 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb7a07-4538-4a8a-9359-41b1a3a710da None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:55.017043 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb7a07-4538-4a8a-9359-41b1a3a710da None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:55.017529 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb7a07-4538-4a8a-9359-41b1a3a710da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:55.024835 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 482/952] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:18:55.036544 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-afeb7a07-4538-4a8a-9359-41b1a3a710da None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:55.038006 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-afeb7a07-4538-4a8a-9359-41b1a3a710da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:55.047266 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e8299-0bf7-4cf7-9ebc-5e378d14d2fb None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:55.112503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 471/953] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:18:55.113772 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bc8b0-b866-4773-b674-d92985fece76 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:55.114057 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bc8b0-b866-4773-b674-d92985fece76 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:55.114329 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bc8b0-b866-4773-b674-d92985fece76 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:55.116419 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 472/954] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:55.121389 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f4269-a9e5-4b24-98d7-bbd30db46567 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:55.121674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f4269-a9e5-4b24-98d7-bbd30db46567 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:55.121949 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f4269-a9e5-4b24-98d7-bbd30db46567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:55.142633 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-5f3f4269-a9e5-4b24-98d7-bbd30db46567 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:55.143819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5f3f4269-a9e5-4b24-98d7-bbd30db46567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:55.146102 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:55.163079 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e8299-0bf7-4cf7-9ebc-5e378d14d2fb None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:55.198484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9178f-8430-40f9-a53d-54f0d7261d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:55.199043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9178f-8430-40f9-a53d-54f0d7261d52 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:55.199317 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9178f-8430-40f9-a53d-54f0d7261d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:55.199588 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9178f-8430-40f9-a53d-54f0d7261d52 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:55.200215 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab9178f-8430-40f9-a53d-54f0d7261d52 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:55.201583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab9178f-8430-40f9-a53d-54f0d7261d52 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:55.210433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-aab9178f-8430-40f9-a53d-54f0d7261d52 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 12:18:55.210433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:55.210433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:18:55.210433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:18:55.210433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:18:55.210433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:18:55.210433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:18:55.210433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:55.210433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 12:18:55.210433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:55.210433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:18:55.210433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:55.210433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:55.210433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:55.210433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:55.210433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:55.210433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:55.211648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:55.211648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:55.211648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:55.211648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:55.211648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:55.211648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:55.211648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:18:55.211648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:18:55.211648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:18:55.211648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:18:55.211648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:55.211648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:55.211648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:55.211648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:55.211648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:55.211648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:55.213246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:55.213246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:55.213246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:55.213246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:55.213246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:55.213246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:55.213246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:55.213246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:55.213246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:55.213246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:55.213246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:18:55.213246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:55.213246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 12:18:55.213246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:55.213246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 483/955] 10.222.0.20 () {56 vars in 1126 bytes} [Sat Aug 6 12:18:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:55.214306 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9178f-8430-40f9-a53d-54f0d7261d52 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:55.220254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 473/956] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:18:55.220972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f4269-a9e5-4b24-98d7-bbd30db46567 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:55.272582 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:55.336014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f4269-a9e5-4b24-98d7-bbd30db46567 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:55.336250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9178f-8430-40f9-a53d-54f0d7261d52 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:55.372508 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229578f-a3f1-445b-bc09-4475796c8b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:55.372704 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d098a5f-5fa7-47ae-b3d3-72d663dcc27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:55.373120 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229578f-a3f1-445b-bc09-4475796c8b66 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:55.373241 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d098a5f-5fa7-47ae-b3d3-72d663dcc27c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:55.373411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229578f-a3f1-445b-bc09-4475796c8b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:55.373516 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d098a5f-5fa7-47ae-b3d3-72d663dcc27c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:55.373664 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229578f-a3f1-445b-bc09-4475796c8b66 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:55.373768 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d098a5f-5fa7-47ae-b3d3-72d663dcc27c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:55.374321 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7229578f-a3f1-445b-bc09-4475796c8b66 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:55.374423 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d098a5f-5fa7-47ae-b3d3-72d663dcc27c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:55.374616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7229578f-a3f1-445b-bc09-4475796c8b66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:55.375730 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d098a5f-5fa7-47ae-b3d3-72d663dcc27c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:55.375959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7229578f-a3f1-445b-bc09-4475796c8b66 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:55.385971 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 474/957] 10.222.0.20 () {56 vars in 1122 bytes} [Sat Aug 6 12:18:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:55.385971 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229578f-a3f1-445b-bc09-4475796c8b66 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:55.386180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-8d098a5f-5fa7-47ae-b3d3-72d663dcc27c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 12:18:55.386180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:55.386180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:18:55.386180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:18:55.386180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:18:55.386180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:18:55.386180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:18:55.386180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:55.386180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 12:18:55.386180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:55.386180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:18:55.386180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:55.386180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:55.386180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:55.386180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:55.386180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:55.386180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:55.387168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:55.387168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:55.387168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:55.387168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:55.387168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:55.387168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:55.387168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:18:55.387168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:18:55.387168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:18:55.387168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:18:55.387168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:55.387168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:55.387168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:55.387168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:55.387168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:55.387168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:55.388077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:55.388077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:55.388077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:55.388077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:55.388077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:55.388077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:55.388077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:55.388077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:55.388077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:55.388077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:55.388077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:18:55.388077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:55.388077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 12:18:55.388077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:55.391650 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 484/958] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:55.392616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d098a5f-5fa7-47ae-b3d3-72d663dcc27c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:55.408541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:55.493057 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229578f-a3f1-445b-bc09-4475796c8b66 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:55.499487 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d098a5f-5fa7-47ae-b3d3-72d663dcc27c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:55.501065 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:55.530702 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229a482-4479-400f-9543-6544c19aec2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:55.531432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229a482-4479-400f-9543-6544c19aec2b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:55.531777 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229a482-4479-400f-9543-6544c19aec2b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:55.532116 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229a482-4479-400f-9543-6544c19aec2b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:55.536260 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f206e-41c7-436f-868b-6659e1b27cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:55.536900 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f206e-41c7-436f-868b-6659e1b27cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:55.537182 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f206e-41c7-436f-868b-6659e1b27cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:55.537457 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f206e-41c7-436f-868b-6659e1b27cb8 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:55.538132 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f206e-41c7-436f-868b-6659e1b27cb8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:55.538434 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f206e-41c7-436f-868b-6659e1b27cb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:55.540342 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f206e-41c7-436f-868b-6659e1b27cb8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:55.545354 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229a482-4479-400f-9543-6544c19aec2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:18:55.550186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 485/959] 10.222.0.20 () {56 vars in 1122 bytes} [Sat Aug 6 12:18:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:55.550186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f206e-41c7-436f-868b-6659e1b27cb8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:55.550839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-b229a482-4479-400f-9543-6544c19aec2b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 12:18:55.550839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:55.550839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:55.550839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:55.550839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:55.550839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:55.550839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:55.550839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:55.550839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:55.550839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:55.550839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:55.550839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:55.550839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:18:55.550839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:18:55.550839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:18:55.550839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:18:55.551944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:18:55.551944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:18:55.551944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:18:55.551944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:18:55.551944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:55.551944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:55.551944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:55.551944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:55.551944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:55.551944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:55.551944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:55.551944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:55.551944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:55.551944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:55.551944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:55.551944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:55.553049 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:55.553049 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:55.553049 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:55.553049 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:55.553049 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:18:55.553049 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:18:55.553049 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:18:55.553049 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:18:55.553049 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:18:55.553049 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:18:55.553049 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 12:18:55.553049 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:55.553049 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 475/960] 10.222.0.20 () {56 vars in 1120 bytes} [Sat Aug 6 12:18:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:55.558919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229a482-4479-400f-9543-6544c19aec2b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:55.637599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:55.660991 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f206e-41c7-436f-868b-6659e1b27cb8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:55.665775 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229a482-4479-400f-9543-6544c19aec2b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:55.697677 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a576804-5f09-4dfc-a27d-6efea201f74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:55.698298 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a576804-5f09-4dfc-a27d-6efea201f74a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:55.698581 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a576804-5f09-4dfc-a27d-6efea201f74a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:55.698861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a576804-5f09-4dfc-a27d-6efea201f74a None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:55.699538 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a576804-5f09-4dfc-a27d-6efea201f74a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:55.699844 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a576804-5f09-4dfc-a27d-6efea201f74a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:55.701242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a576804-5f09-4dfc-a27d-6efea201f74a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:55.701344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d4d03-abb0-445b-802d-339742c00abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:55.701848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d4d03-abb0-445b-802d-339742c00abc None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:55.702219 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d4d03-abb0-445b-802d-339742c00abc None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:55.702395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d4d03-abb0-445b-802d-339742c00abc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:55.703097 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7d4d03-abb0-445b-802d-339742c00abc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:55.704514 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7d4d03-abb0-445b-802d-339742c00abc None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:55.706853 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 486/961] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Aug 6 12:18:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:55.707887 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a576804-5f09-4dfc-a27d-6efea201f74a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:55.730497 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 476/962] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:18:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:18:55.731559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d4d03-abb0-445b-802d-339742c00abc None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:55.768814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:55.870824 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a576804-5f09-4dfc-a27d-6efea201f74a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:55.881017 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d4d03-abb0-445b-802d-339742c00abc None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:55.913817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf88181-2439-49cf-a825-1ec35e843a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:55.914543 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf88181-2439-49cf-a825-1ec35e843a8e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:55.914886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf88181-2439-49cf-a825-1ec35e843a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:55.915235 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf88181-2439-49cf-a825-1ec35e843a8e None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:55.916027 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf88181-2439-49cf-a825-1ec35e843a8e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:55.916452 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf88181-2439-49cf-a825-1ec35e843a8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:55.917884 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf88181-2439-49cf-a825-1ec35e843a8e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:55.925889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf5396-5df6-475e-8682-57948ae9521e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:55.927002 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf5396-5df6-475e-8682-57948ae9521e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:55.927440 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf5396-5df6-475e-8682-57948ae9521e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:55.927860 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf5396-5df6-475e-8682-57948ae9521e None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:55.928825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 487/963] 10.222.0.20 () {56 vars in 1115 bytes} [Sat Aug 6 12:18:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:55.929758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf5396-5df6-475e-8682-57948ae9521e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:55.929758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf5396-5df6-475e-8682-57948ae9521e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:55.930755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf5396-5df6-475e-8682-57948ae9521e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:55.935725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 477/964] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Aug 6 12:18:55 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:55.941481 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf88181-2439-49cf-a825-1ec35e843a8e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:55.947215 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:56.062271 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf88181-2439-49cf-a825-1ec35e843a8e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:56.099180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f78b782-eae4-4886-b6d4-814b66090590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:56.099823 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78b782-eae4-4886-b6d4-814b66090590 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:56.100108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78b782-eae4-4886-b6d4-814b66090590 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:56.100408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78b782-eae4-4886-b6d4-814b66090590 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:56.111798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f78b782-eae4-4886-b6d4-814b66090590 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:56.113282 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f78b782-eae4-4886-b6d4-814b66090590 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:56.129476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-9f78b782-eae4-4886-b6d4-814b66090590 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:56.129476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:56.129476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:56.129476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:56.129476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:56.129476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:56.129476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:56.129476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:56.129476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:56.129476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:56.129476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:56.129476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:56.129476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:56.129476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:56.129476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:56.130456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:56.130456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:56.130456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:56.130456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:56.130456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:56.130456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:56.130456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:56.130456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:56.130456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:56.130456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:56.130456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:56.130456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:56.130456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:56.130456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:56.130456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:56.130456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:56.130456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:56.131355 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:56.131355 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:56.131355 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:56.131355 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:56.131355 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 488/965] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:56.136892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf5396-5df6-475e-8682-57948ae9521e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:56.210624 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029145f-fccc-47ef-bcc1-544684dcd20a None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:56.210954 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029145f-fccc-47ef-bcc1-544684dcd20a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:56.211180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029145f-fccc-47ef-bcc1-544684dcd20a None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:56.212855 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 489/966] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:18:56.218747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad45fc2-b261-4a64-b826-5133ce1842b6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:56.219051 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad45fc2-b261-4a64-b826-5133ce1842b6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:56.219369 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad45fc2-b261-4a64-b826-5133ce1842b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:56.223634 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:56.243673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-fad45fc2-b261-4a64-b826-5133ce1842b6 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:56.244908 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fad45fc2-b261-4a64-b826-5133ce1842b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:56.245797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf5396-5df6-475e-8682-57948ae9521e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:56.283218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3aeabf-f100-462b-a295-819f1a149a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:56.283842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3aeabf-f100-462b-a295-819f1a149a4f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:56.284121 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3aeabf-f100-462b-a295-819f1a149a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:56.284429 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3aeabf-f100-462b-a295-819f1a149a4f None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:56.285137 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3aeabf-f100-462b-a295-819f1a149a4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:56.285429 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3aeabf-f100-462b-a295-819f1a149a4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:56.286807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3aeabf-f100-462b-a295-819f1a149a4f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:56.302226 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 478/967] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:56.308249 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3aeabf-f100-462b-a295-819f1a149a4f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:56.312379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:56.329513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 490/968] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 12:18:56.355953 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad45fc2-b261-4a64-b826-5133ce1842b6 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:56.431077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3aeabf-f100-462b-a295-819f1a149a4f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:56.472575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad45fc2-b261-4a64-b826-5133ce1842b6 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:56.474987 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef105e7d-a2ff-458e-88b9-d311e45dcccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:56.476494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef105e7d-a2ff-458e-88b9-d311e45dcccf None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:56.476781 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef105e7d-a2ff-458e-88b9-d311e45dcccf None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:56.477062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef105e7d-a2ff-458e-88b9-d311e45dcccf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:56.485498 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef105e7d-a2ff-458e-88b9-d311e45dcccf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:56.487563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef105e7d-a2ff-458e-88b9-d311e45dcccf None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:56.503233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-ef105e7d-a2ff-458e-88b9-d311e45dcccf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 12:18:56.503233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:56.503233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:56.503233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:56.503233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:56.503233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:56.503233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:56.503233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:56.503233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:56.503233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:56.503233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:56.503233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:56.503233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:56.503233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:56.503233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:56.503233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:56.504344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:56.504344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:56.504344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:56.504344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:56.504344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:56.504344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:56.504344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:56.504344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:56.504344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:56.504344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:56.504344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:56.504344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:56.504344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:56.504344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:56.504344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:56.504344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:56.504344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:56.504344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:56.510031 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:56.510031 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:56.510031 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:56.510031 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:56.510031 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:56.510031 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:56.510031 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:56.510031 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:56.510031 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 06 12:18:56.510031 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:18:56.510031 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 479/969] 10.222.0.20 () {56 vars in 1117 bytes} [Sat Aug 6 12:18:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:56.513211 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef105e7d-a2ff-458e-88b9-d311e45dcccf None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:56.516115 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:56.518287 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f02499-052f-43e3-9111-4850cdda48a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:56.518429 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f02499-052f-43e3-9111-4850cdda48a0 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:56.518666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f02499-052f-43e3-9111-4850cdda48a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:56.519018 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f02499-052f-43e3-9111-4850cdda48a0 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:56.519758 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f02499-052f-43e3-9111-4850cdda48a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:56.521308 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f02499-052f-43e3-9111-4850cdda48a0 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:56.538276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.sql.core [None req-e8f02499-052f-43e3-9111-4850cdda48a0 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 06 12:18:56.538276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 06 12:18:56.538276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [parameters: {'user_id': '81dc3badfeb24127b40742034fc6e98d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 06 12:18:56.538276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80144) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 06 12:18:56.545805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-e8f02499-052f-43e3-9111-4850cdda48a0 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 06 12:18:56.545805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:56.545805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 12:18:56.545805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 12:18:56.545805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 12:18:56.545805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 12:18:56.545805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 12:18:56.545805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 12:18:56.545805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 12:18:56.545805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application conn.query(q) Aug 06 12:18:56.545805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 12:18:56.545805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 12:18:56.545805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 12:18:56.545805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application result.read() Aug 06 12:18:56.545805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 12:18:56.546760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 12:18:56.546760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 12:18:56.546760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 12:18:56.546760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 12:18:56.546760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 12:18:56.546760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 12:18:56.546760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 12:18:56.546760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 06 12:18:56.546760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:56.546760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 06 12:18:56.546760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:56.546760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:56.546760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 06 12:18:56.546760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 06 12:18:56.546760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 06 12:18:56.546760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 06 12:18:56.546760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 12:18:56.546760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application next(self.gen) Aug 06 12:18:56.547833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 06 12:18:56.547833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application yield resource Aug 06 12:18:56.547833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 06 12:18:56.547833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application next(self.gen) Aug 06 12:18:56.547833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 06 12:18:56.547833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.session.rollback() Aug 06 12:18:56.547833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 06 12:18:56.547833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.force_reraise() Aug 06 12:18:56.547833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 06 12:18:56.547833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise self.value Aug 06 12:18:56.547833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 06 12:18:56.547833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 06 12:18:56.547833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 06 12:18:56.547833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application session.commit() Aug 06 12:18:56.547833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 06 12:18:56.547833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 06 12:18:56.547833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 06 12:18:56.548804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._prepare_impl() Aug 06 12:18:56.548804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 06 12:18:56.548804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.session.flush() Aug 06 12:18:56.548804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 06 12:18:56.548804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._flush(objects) Aug 06 12:18:56.548804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 06 12:18:56.548804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 06 12:18:56.548804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 06 12:18:56.548804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application compat.raise_( Aug 06 12:18:56.548804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 12:18:56.548804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception Aug 06 12:18:56.548804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 06 12:18:56.548804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application flush_context.execute() Aug 06 12:18:56.548804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 06 12:18:56.548804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rec.execute(self) Aug 06 12:18:56.548804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 06 12:18:56.548804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 06 12:18:56.549737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 06 12:18:56.549737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 06 12:18:56.549737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 06 12:18:56.549737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 06 12:18:56.549737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 06 12:18:56.549737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 06 12:18:56.549737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 06 12:18:56.549737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 06 12:18:56.549737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 06 12:18:56.549737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 06 12:18:56.549737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 06 12:18:56.549737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 06 12:18:56.549737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 06 12:18:56.549737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 06 12:18:56.549737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 06 12:18:56.549737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception Aug 06 12:18:56.550652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 06 12:18:56.550652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 06 12:18:56.550652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 06 12:18:56.550652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 06 12:18:56.550652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 06 12:18:56.550652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application result = self._query(query) Aug 06 12:18:56.550652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 06 12:18:56.550652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application conn.query(q) Aug 06 12:18:56.550652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 06 12:18:56.550652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 06 12:18:56.550652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 06 12:18:56.550652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application result.read() Aug 06 12:18:56.550652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 06 12:18:56.550652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 06 12:18:56.550652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 06 12:18:56.550652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 06 12:18:56.551484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 06 12:18:56.551484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 06 12:18:56.551484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 06 12:18:56.551484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 06 12:18:56.551484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 06 12:18:56.551484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 06 12:18:56.551484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [parameters: {'user_id': '81dc3badfeb24127b40742034fc6e98d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 06 12:18:56.551484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 06 12:18:56.551484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:56.551484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:18:56.551484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:56.551484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:56.551484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:56.551484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:56.551484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:56.554290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:56.554290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:56.554290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:56.554290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:56.554290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:56.554290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:56.554290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:56.554290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 06 12:18:56.554290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 06 12:18:56.554290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:56.554290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:56.554290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:56.554290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:56.554290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:56.554290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:56.554290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 06 12:18:56.554290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 06 12:18:56.556468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 06 12:18:56.556468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 06 12:18:56.556468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 06 12:18:56.556468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 06 12:18:56.556468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 06 12:18:56.556468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:56.556468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 491/970] 10.222.0.20 () {60 vars in 1131 bytes} [Sat Aug 6 12:18:56 2022] POST /identity/v3/users => generated 160 bytes in 192 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:18:56.556468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f02499-052f-43e3-9111-4850cdda48a0 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:56.609559 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:56.641752 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef105e7d-a2ff-458e-88b9-d311e45dcccf None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:56.668588 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f02499-052f-43e3-9111-4850cdda48a0 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:56.685837 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c84d88-e090-4132-96ee-adde270a4f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:56.686853 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c84d88-e090-4132-96ee-adde270a4f0e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:56.687490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c84d88-e090-4132-96ee-adde270a4f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:56.688063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c84d88-e090-4132-96ee-adde270a4f0e None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:56.689093 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c84d88-e090-4132-96ee-adde270a4f0e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:56.689615 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c84d88-e090-4132-96ee-adde270a4f0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:56.691383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c84d88-e090-4132-96ee-adde270a4f0e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:56.706297 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b15280-3047-4b48-b4bd-4bb4bca88f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:56.706937 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b15280-3047-4b48-b4bd-4bb4bca88f71 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:56.707219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b15280-3047-4b48-b4bd-4bb4bca88f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:56.707496 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b15280-3047-4b48-b4bd-4bb4bca88f71 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:56.716395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b15280-3047-4b48-b4bd-4bb4bca88f71 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:56.716644 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 480/971] 10.222.0.20 () {56 vars in 1149 bytes} [Sat Aug 6 12:18:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:56.717131 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b15280-3047-4b48-b4bd-4bb4bca88f71 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:56.717696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c84d88-e090-4132-96ee-adde270a4f0e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:56.729697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-79b15280-3047-4b48-b4bd-4bb4bca88f71 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 12:18:56.729697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:56.729697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:56.729697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:56.729697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:56.729697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:56.729697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:56.729697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:56.729697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:56.729697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:56.729697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:56.729697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:56.729697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:18:56.729697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:18:56.729697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:18:56.729697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:18:56.730892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:56.730892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:56.730892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:18:56.730892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:56.730892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:18:56.730892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:18:56.730892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:56.730892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:56.730892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:56.730892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:56.730892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:56.730892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:56.730892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:56.730892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:56.730892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:56.730892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:56.730892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:56.730892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:56.732068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:56.732068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:56.732068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:18:56.732068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:18:56.732068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:18:56.732068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:18:56.732068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:18:56.732068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:18:56.732068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 06 12:18:56.732068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:56.738024 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 492/972] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:18:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:56.738905 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b15280-3047-4b48-b4bd-4bb4bca88f71 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:56.783425 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:56.828038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c84d88-e090-4132-96ee-adde270a4f0e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:56.848758 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b15280-3047-4b48-b4bd-4bb4bca88f71 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:56.859100 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:56.868067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9dd639-5b4f-4a50-80f2-448521286f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:56.869159 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9dd639-5b4f-4a50-80f2-448521286f9f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:56.869589 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9dd639-5b4f-4a50-80f2-448521286f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:56.870352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9dd639-5b4f-4a50-80f2-448521286f9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:56.887754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89e4c1-3138-4843-9335-5233c6e149fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:56.888555 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89e4c1-3138-4843-9335-5233c6e149fc None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:56.888948 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89e4c1-3138-4843-9335-5233c6e149fc None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:56.889325 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89e4c1-3138-4843-9335-5233c6e149fc None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:56.890186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c89e4c1-3138-4843-9335-5233c6e149fc None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:56.890640 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c89e4c1-3138-4843-9335-5233c6e149fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:56.892173 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c89e4c1-3138-4843-9335-5233c6e149fc None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:56.941218 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 493/973] 10.222.0.20 () {56 vars in 1101 bytes} [Sat Aug 6 12:18:56 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:56.951191 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9dd639-5b4f-4a50-80f2-448521286f9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:56.953360 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9dd639-5b4f-4a50-80f2-448521286f9f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:57.034805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:57.082236 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 481/974] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:18:56 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 12:18:57.090218 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89e4c1-3138-4843-9335-5233c6e149fc None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:57.199253 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89e4c1-3138-4843-9335-5233c6e149fc None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:57.234975 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0b0b3-3eec-4669-9ad9-a270e0d9a810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:57.235869 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0b0b3-3eec-4669-9ad9-a270e0d9a810 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:57.236296 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0b0b3-3eec-4669-9ad9-a270e0d9a810 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:57.236748 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0b0b3-3eec-4669-9ad9-a270e0d9a810 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:57.247485 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0b0b3-3eec-4669-9ad9-a270e0d9a810 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:57.249119 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0b0b3-3eec-4669-9ad9-a270e0d9a810 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:57.264647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-e7e0b0b3-3eec-4669-9ad9-a270e0d9a810 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:57.264647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:57.264647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:57.264647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:57.264647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:57.264647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:57.264647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:57.264647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:57.264647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:57.264647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:57.264647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:57.264647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:57.264647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:18:57.264647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:18:57.264647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:18:57.265735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:18:57.265735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:57.265735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:57.265735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:57.265735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:57.265735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:57.265735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:57.265735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:57.265735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:57.265735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:57.265735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:57.265735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:57.265735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:57.265735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:57.265735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:57.265735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:57.265735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:57.266845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:18:57.266845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:18:57.266845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:18:57.266845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:57.268633 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 494/975] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:18:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:57.274912 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9dd639-5b4f-4a50-80f2-448521286f9f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:57.378013 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9dd639-5b4f-4a50-80f2-448521286f9f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:57.413508 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e078d60-5048-4499-92dd-c8ae0181b0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:57.414332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e078d60-5048-4499-92dd-c8ae0181b0af None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:57.414759 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e078d60-5048-4499-92dd-c8ae0181b0af None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:57.415210 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e078d60-5048-4499-92dd-c8ae0181b0af None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:57.416036 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e078d60-5048-4499-92dd-c8ae0181b0af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:57.416520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e078d60-5048-4499-92dd-c8ae0181b0af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:57.418040 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e078d60-5048-4499-92dd-c8ae0181b0af None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:57.430604 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 482/976] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:57.436335 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0b0b3-3eec-4669-9ad9-a270e0d9a810 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:57.457345 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:57.541751 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0b0b3-3eec-4669-9ad9-a270e0d9a810 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:57.578746 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce2d51-2c9f-4966-a314-3246b9e9e26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:57.579388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce2d51-2c9f-4966-a314-3246b9e9e26b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:57.579674 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce2d51-2c9f-4966-a314-3246b9e9e26b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:57.579948 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce2d51-2c9f-4966-a314-3246b9e9e26b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:57.586945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce2d51-2c9f-4966-a314-3246b9e9e26b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:57.588467 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce2d51-2c9f-4966-a314-3246b9e9e26b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:57.599829 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-5fce2d51-2c9f-4966-a314-3246b9e9e26b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:57.599829 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:57.599829 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:57.599829 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:57.599829 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:57.599829 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:57.599829 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:57.599829 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:57.599829 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:57.599829 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:57.599829 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:57.599829 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:57.599829 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:18:57.599829 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:18:57.599829 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:18:57.600956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:18:57.600956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:57.600956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:57.600956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:57.600956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:57.600956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:57.600956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:57.600956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:57.600956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:57.600956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:57.600956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:57.600956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:57.600956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:57.600956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:57.600956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:57.600956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:57.600956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:57.601863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:18:57.601863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:18:57.601863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:18:57.601863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:18:57.601863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:18:57.601863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:18:57.601863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:18:57.601863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:57.601863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 495/977] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:18:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:57.608463 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e078d60-5048-4499-92dd-c8ae0181b0af None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:57.633101 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:57.713659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e078d60-5048-4499-92dd-c8ae0181b0af None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:57.747407 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28e59e-eb1d-4f38-8677-2c9d4ad13f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:57.748021 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28e59e-eb1d-4f38-8677-2c9d4ad13f2c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:57.748392 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28e59e-eb1d-4f38-8677-2c9d4ad13f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:57.748669 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28e59e-eb1d-4f38-8677-2c9d4ad13f2c None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:57.749415 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28e59e-eb1d-4f38-8677-2c9d4ad13f2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:57.749720 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28e59e-eb1d-4f38-8677-2c9d4ad13f2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:57.751140 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28e59e-eb1d-4f38-8677-2c9d4ad13f2c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:57.764463 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 483/978] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:18:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:57.769925 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce2d51-2c9f-4966-a314-3246b9e9e26b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:57.852198 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:57.867625 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a9892-9d17-4c90-b65a-e9253f3dedb9 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:57.868087 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a9892-9d17-4c90-b65a-e9253f3dedb9 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:57.868325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a9892-9d17-4c90-b65a-e9253f3dedb9 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:57.869772 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 484/979] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:57.874594 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae065b-9982-4946-9c0c-9d5db1cfb29a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:57.874880 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae065b-9982-4946-9c0c-9d5db1cfb29a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:57.875163 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae065b-9982-4946-9c0c-9d5db1cfb29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:57.878048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce2d51-2c9f-4966-a314-3246b9e9e26b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:57.895729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-5aae065b-9982-4946-9c0c-9d5db1cfb29a None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:57.897306 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5aae065b-9982-4946-9c0c-9d5db1cfb29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:57.917167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a25162-c4fc-48c0-9f89-276dcbb5903a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:57.918118 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a25162-c4fc-48c0-9f89-276dcbb5903a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:57.918697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a25162-c4fc-48c0-9f89-276dcbb5903a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:57.918972 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a25162-c4fc-48c0-9f89-276dcbb5903a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:57.919767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a25162-c4fc-48c0-9f89-276dcbb5903a None admin] RBAC: Adding query filter params (role.id=ed864363c041447ab55d1795a8c7092b, scope.project.id=13fba33c46cf40bbb15a08c690b4c98c, user.id=c810b52785b94c0cab4fd9f0d6e65825) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:57.920077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a25162-c4fc-48c0-9f89-276dcbb5903a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:57.921705 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a25162-c4fc-48c0-9f89-276dcbb5903a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:57.928179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 496/980] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:18:57 2022] GET /identity/v3/role_assignments?user.id=c810b52785b94c0cab4fd9f0d6e65825&scope.project.id=13fba33c46cf40bbb15a08c690b4c98c&role.id=ed864363c041447ab55d1795a8c7092b => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:57.980335 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 485/981] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 12:18:58.002862 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a25162-c4fc-48c0-9f89-276dcbb5903a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:58.015720 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7843a3d-bc0b-4ac4-843b-145b34fe2707 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:58.016004 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7843a3d-bc0b-4ac4-843b-145b34fe2707 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:58.016286 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7843a3d-bc0b-4ac4-843b-145b34fe2707 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:58.018024 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 486/982] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:18:58.023934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ac71e-52b3-43cf-8b24-a1ce16c7cf14 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:58.024268 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ac71e-52b3-43cf-8b24-a1ce16c7cf14 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:58.024570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ac71e-52b3-43cf-8b24-a1ce16c7cf14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:58.044412 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3e6ac71e-52b3-43cf-8b24-a1ce16c7cf14 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:58.045639 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3e6ac71e-52b3-43cf-8b24-a1ce16c7cf14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:18:58.049669 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:58.126918 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 487/983] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 12:18:58.130324 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a25162-c4fc-48c0-9f89-276dcbb5903a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:58.149778 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ac71e-52b3-43cf-8b24-a1ce16c7cf14 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:58.166008 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000d03a-02cd-4f1d-bc48-42c9638a6c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:58.166642 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000d03a-02cd-4f1d-bc48-42c9638a6c76 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:58.166919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000d03a-02cd-4f1d-bc48-42c9638a6c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:58.167186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000d03a-02cd-4f1d-bc48-42c9638a6c76 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:58.167804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000d03a-02cd-4f1d-bc48-42c9638a6c76 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:58.169180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000d03a-02cd-4f1d-bc48-42c9638a6c76 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:58.178334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-3000d03a-02cd-4f1d-bc48-42c9638a6c76 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 12:18:58.178334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:58.178334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:18:58.178334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:18:58.178334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:18:58.178334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:18:58.178334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:18:58.178334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:58.178334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 12:18:58.178334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:58.178334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:18:58.178334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:58.178334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:18:58.178334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:18:58.178334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:18:58.178334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:18:58.178334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:18:58.180022 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:18:58.180022 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:18:58.180022 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:18:58.180022 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:18:58.180022 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:18:58.180022 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:18:58.180022 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:18:58.180022 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:18:58.180022 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:18:58.180022 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:18:58.180022 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:18:58.180022 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:18:58.180022 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:18:58.180022 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:18:58.180022 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:18:58.180022 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:18:58.180984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:18:58.180984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:18:58.180984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:18:58.180984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:18:58.180984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:18:58.180984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:18:58.180984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:18:58.180984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:18:58.180984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:18:58.180984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:18:58.180984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:18:58.180984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:18:58.180984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 12:18:58.180984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:18:58.180984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 497/984] 10.222.0.20 () {56 vars in 1126 bytes} [Sat Aug 6 12:18:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:18:58.185679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000d03a-02cd-4f1d-bc48-42c9638a6c76 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:58.196706 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:58.262884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ac71e-52b3-43cf-8b24-a1ce16c7cf14 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:58.294604 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000d03a-02cd-4f1d-bc48-42c9638a6c76 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:58.297554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badbaaa2-f254-44ff-86c2-71d20a0ee0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:58.298175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbaaa2-f254-44ff-86c2-71d20a0ee0cc None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:58.298456 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbaaa2-f254-44ff-86c2-71d20a0ee0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:58.298732 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbaaa2-f254-44ff-86c2-71d20a0ee0cc None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:58.299428 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badbaaa2-f254-44ff-86c2-71d20a0ee0cc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:58.300839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badbaaa2-f254-44ff-86c2-71d20a0ee0cc None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:58.311750 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:58.319484 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 488/985] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Aug 6 12:18:58 2022] POST /identity/v3/services => generated 242 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 12:18:58.341516 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7c253-d078-4ad8-8ae7-e835cf3ab731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:58.342161 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7c253-d078-4ad8-8ae7-e835cf3ab731 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:58.342445 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7c253-d078-4ad8-8ae7-e835cf3ab731 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:58.342781 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7c253-d078-4ad8-8ae7-e835cf3ab731 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:58.343488 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7c253-d078-4ad8-8ae7-e835cf3ab731 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:58.343794 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7c253-d078-4ad8-8ae7-e835cf3ab731 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:58.345245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7c253-d078-4ad8-8ae7-e835cf3ab731 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:58.351417 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 498/986] 10.222.0.20 () {56 vars in 1122 bytes} [Sat Aug 6 12:18:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:18:58.357265 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badbaaa2-f254-44ff-86c2-71d20a0ee0cc None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:58.473419 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badbaaa2-f254-44ff-86c2-71d20a0ee0cc None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:58.510668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09780192-3a72-4fc7-8297-da8dd5ce0112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:58.511284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09780192-3a72-4fc7-8297-da8dd5ce0112 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:58.511569 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09780192-3a72-4fc7-8297-da8dd5ce0112 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:58.511933 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09780192-3a72-4fc7-8297-da8dd5ce0112 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:58.512645 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09780192-3a72-4fc7-8297-da8dd5ce0112 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:58.513006 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09780192-3a72-4fc7-8297-da8dd5ce0112 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:58.514329 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09780192-3a72-4fc7-8297-da8dd5ce0112 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:58.519778 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 489/987] 10.222.0.20 () {56 vars in 1122 bytes} [Sat Aug 6 12:18:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:58.525751 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7c253-d078-4ad8-8ae7-e835cf3ab731 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:58.592712 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:58.629443 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7c253-d078-4ad8-8ae7-e835cf3ab731 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:58.663908 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3ffc7-b8ac-4bd9-9d60-189bea515562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:58.664558 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3ffc7-b8ac-4bd9-9d60-189bea515562 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:58.664837 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3ffc7-b8ac-4bd9-9d60-189bea515562 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:58.665106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3ffc7-b8ac-4bd9-9d60-189bea515562 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:58.665859 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c3ffc7-b8ac-4bd9-9d60-189bea515562 None admin] RBAC: Adding query filter params (interface=admin, service_id=8de0bfb4caab448a8de667770f89657c, region_id=RegionOne) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:18:58.666161 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c3ffc7-b8ac-4bd9-9d60-189bea515562 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:58.667503 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c3ffc7-b8ac-4bd9-9d60-189bea515562 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:58.672744 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 499/988] 10.222.0.20 () {56 vars in 1249 bytes} [Sat Aug 6 12:18:58 2022] GET /identity/v3/endpoints?service_id=8de0bfb4caab448a8de667770f89657c&interface=admin®ion_id=RegionOne => generated 193 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:18:58.678358 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09780192-3a72-4fc7-8297-da8dd5ce0112 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:18:58.741941 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:18:58.781171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09780192-3a72-4fc7-8297-da8dd5ce0112 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:18:58.814058 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bebd08-33fa-4701-99bb-6ee03a8a1f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:18:58.814833 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bebd08-33fa-4701-99bb-6ee03a8a1f3a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:58.815115 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bebd08-33fa-4701-99bb-6ee03a8a1f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:58.815389 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bebd08-33fa-4701-99bb-6ee03a8a1f3a None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:58.816138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bebd08-33fa-4701-99bb-6ee03a8a1f3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:18:58.817607 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bebd08-33fa-4701-99bb-6ee03a8a1f3a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:18:58.828662 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 490/989] 10.222.0.20 () {56 vars in 1087 bytes} [Sat Aug 6 12:18:58 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:18:59.966796 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697babc-ce9b-4689-b6b2-622a9ca19004 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:59.967325 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697babc-ce9b-4689-b6b2-622a9ca19004 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:59.967325 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697babc-ce9b-4689-b6b2-622a9ca19004 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:59.968890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 500/990] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:18:59.973754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9575bdf-5ce8-4202-bbe5-987152bccc6a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:18:59.974129 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9575bdf-5ce8-4202-bbe5-987152bccc6a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:18:59.974323 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9575bdf-5ce8-4202-bbe5-987152bccc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:18:59.996713 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-d9575bdf-5ce8-4202-bbe5-987152bccc6a None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:18:59.998531 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d9575bdf-5ce8-4202-bbe5-987152bccc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:00.067648 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 491/991] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:18:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:19:00.090273 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697babc-ce9b-4689-b6b2-622a9ca19004 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:00.193735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697babc-ce9b-4689-b6b2-622a9ca19004 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:00.222672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:00.230121 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea883f9b-5325-4e15-9edc-3af617a176cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:00.230764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea883f9b-5325-4e15-9edc-3af617a176cb None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:00.231047 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea883f9b-5325-4e15-9edc-3af617a176cb None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:00.231322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea883f9b-5325-4e15-9edc-3af617a176cb None admin] PATH_INFO: `/v3/services/image` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:00.231948 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea883f9b-5325-4e15-9edc-3af617a176cb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:00.233345 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea883f9b-5325-4e15-9edc-3af617a176cb None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:00.242698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-ea883f9b-5325-4e15-9edc-3af617a176cb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 12:19:00.242698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:00.242698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:19:00.242698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:19:00.242698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:19:00.242698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:19:00.242698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:19:00.242698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:19:00.242698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 12:19:00.242698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:19:00.242698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:19:00.242698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:19:00.242698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:00.242698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:00.242698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:00.242698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:00.242698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:00.243702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:00.243702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:00.243702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:00.243702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:00.243702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:00.243702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:00.243702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:19:00.243702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:19:00.243702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:19:00.243702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:19:00.243702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:00.243702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:00.243702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:00.243702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:00.243702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:00.243702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:00.244600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:00.244600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:00.244600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:00.244600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:00.244600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:00.244600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:00.244600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:00.244600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:00.244600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:00.244600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:00.244600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:19:00.244600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:19:00.244600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 12:19:00.244600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:19:00.245972 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 501/992] 10.222.0.20 () {56 vars in 1105 bytes} [Sat Aug 6 12:19:00 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:19:00.249851 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9575bdf-5ce8-4202-bbe5-987152bccc6a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:00.309305 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:00.333585 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec386b-f9a5-44de-9f49-8a918dd7aa00 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:00.334840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec386b-f9a5-44de-9f49-8a918dd7aa00 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:00.335789 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec386b-f9a5-44de-9f49-8a918dd7aa00 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:00.337668 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 502/993] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:19:00.342698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ae530-9d96-41e7-88c3-00a64e2d925d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:00.342981 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ae530-9d96-41e7-88c3-00a64e2d925d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:00.343256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ae530-9d96-41e7-88c3-00a64e2d925d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:00.354368 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9575bdf-5ce8-4202-bbe5-987152bccc6a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:00.361191 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-669ae530-9d96-41e7-88c3-00a64e2d925d None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:00.362795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-669ae530-9d96-41e7-88c3-00a64e2d925d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:00.393157 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa7f1c-26b2-4057-9927-26cd1ef4c35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:00.393777 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa7f1c-26b2-4057-9927-26cd1ef4c35b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:00.394054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa7f1c-26b2-4057-9927-26cd1ef4c35b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:00.394328 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa7f1c-26b2-4057-9927-26cd1ef4c35b None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:00.395000 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa7f1c-26b2-4057-9927-26cd1ef4c35b None admin] RBAC: Adding query filter params (name=image) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:00.395298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa7f1c-26b2-4057-9927-26cd1ef4c35b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:00.396797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa7f1c-26b2-4057-9927-26cd1ef4c35b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:00.401973 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 492/994] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:19:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:00.411135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa7f1c-26b2-4057-9927-26cd1ef4c35b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:00.436069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 503/995] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 12:19:00.458624 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ae530-9d96-41e7-88c3-00a64e2d925d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:00.522130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa7f1c-26b2-4057-9927-26cd1ef4c35b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:00.546112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:00.559458 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e4eac-c80e-4834-8b49-f0b4eb43f9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:00.560216 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e4eac-c80e-4834-8b49-f0b4eb43f9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:00.560522 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e4eac-c80e-4834-8b49-f0b4eb43f9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:00.561152 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e4eac-c80e-4834-8b49-f0b4eb43f9a1 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:00.561835 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e4eac-c80e-4834-8b49-f0b4eb43f9a1 None admin] RBAC: Adding query filter params (type=image) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:00.562141 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e4eac-c80e-4834-8b49-f0b4eb43f9a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:00.563677 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e4eac-c80e-4834-8b49-f0b4eb43f9a1 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:00.566648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ae530-9d96-41e7-88c3-00a64e2d925d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:00.568131 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 493/996] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:19:00 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:00.603126 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:00.615559 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972abaa-1c3e-415c-ba10-4fe70523d85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:00.617172 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972abaa-1c3e-415c-ba10-4fe70523d85e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:00.617520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972abaa-1c3e-415c-ba10-4fe70523d85e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:00.617863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972abaa-1c3e-415c-ba10-4fe70523d85e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:00.618704 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972abaa-1c3e-415c-ba10-4fe70523d85e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:00.620285 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972abaa-1c3e-415c-ba10-4fe70523d85e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:00.631221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-9972abaa-1c3e-415c-ba10-4fe70523d85e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 12:19:00.631221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:00.631221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:19:00.631221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:19:00.631221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:19:00.631221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:19:00.631221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:19:00.631221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:19:00.631221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 12:19:00.631221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:19:00.631221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:19:00.631221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:19:00.631221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:00.631221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:00.631221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:00.631221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:00.631221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:00.633179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:00.633179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:00.633179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:00.633179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:00.633179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:00.633179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:00.633179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:19:00.633179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:19:00.633179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:19:00.633179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:19:00.633179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:00.633179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:00.633179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:00.633179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:00.633179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:00.633179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:00.634058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:00.634058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:00.634058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:00.634058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:00.634058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:00.634058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:00.634058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:00.634058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:00.634058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:00.634058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:00.634058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:19:00.634058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:19:00.634058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 12:19:00.634058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:19:00.638420 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 504/997] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:19:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:19:00.639796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e4eac-c80e-4834-8b49-f0b4eb43f9a1 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:00.749558 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e4eac-c80e-4834-8b49-f0b4eb43f9a1 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:00.785612 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab438f-5666-4e20-a34a-2a32520f0b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:00.786233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab438f-5666-4e20-a34a-2a32520f0b39 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:00.786573 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab438f-5666-4e20-a34a-2a32520f0b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:00.786782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab438f-5666-4e20-a34a-2a32520f0b39 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:00.787456 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab438f-5666-4e20-a34a-2a32520f0b39 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:00.787756 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab438f-5666-4e20-a34a-2a32520f0b39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:00.789733 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab438f-5666-4e20-a34a-2a32520f0b39 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:00.799089 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 494/998] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Aug 6 12:19:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:00.800811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972abaa-1c3e-415c-ba10-4fe70523d85e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:00.829427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c50c2a-133e-43ab-92bd-4a37f175ee7b None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:00.829791 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c50c2a-133e-43ab-92bd-4a37f175ee7b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:00.829983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c50c2a-133e-43ab-92bd-4a37f175ee7b None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:00.831330 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 495/999] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:19:00.836244 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36fd4e-0ec9-41ff-885f-1a8c21cce845 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:00.836563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36fd4e-0ec9-41ff-885f-1a8c21cce845 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:00.836838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36fd4e-0ec9-41ff-885f-1a8c21cce845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:00.857472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-6e36fd4e-0ec9-41ff-885f-1a8c21cce845 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:00.860015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6e36fd4e-0ec9-41ff-885f-1a8c21cce845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:00.912869 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:00.918894 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972abaa-1c3e-415c-ba10-4fe70523d85e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:00.943689 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 496/1000] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 12:19:00.956695 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cedef-feed-494d-b267-cb8f175439f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:00.957322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cedef-feed-494d-b267-cb8f175439f4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:00.957599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cedef-feed-494d-b267-cb8f175439f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:00.957871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cedef-feed-494d-b267-cb8f175439f4 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:00.958140 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36fd4e-0ec9-41ff-885f-1a8c21cce845 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:00.958547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1cedef-feed-494d-b267-cb8f175439f4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:00.958844 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1cedef-feed-494d-b267-cb8f175439f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:00.960232 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1cedef-feed-494d-b267-cb8f175439f4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:00.968650 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 505/1001] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Aug 6 12:19:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:00.971914 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cedef-feed-494d-b267-cb8f175439f4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:01.039508 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:01.073391 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36fd4e-0ec9-41ff-885f-1a8c21cce845 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:01.100767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cedef-feed-494d-b267-cb8f175439f4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:01.111628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43901e-cbaa-48fc-96b2-080a99cf2c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:01.112277 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43901e-cbaa-48fc-96b2-080a99cf2c2d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:01.112591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43901e-cbaa-48fc-96b2-080a99cf2c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:01.112922 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43901e-cbaa-48fc-96b2-080a99cf2c2d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:01.113560 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43901e-cbaa-48fc-96b2-080a99cf2c2d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:01.114975 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43901e-cbaa-48fc-96b2-080a99cf2c2d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:01.125454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-3b43901e-cbaa-48fc-96b2-080a99cf2c2d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 12:19:01.125454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:01.125454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:19:01.125454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:19:01.125454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:19:01.125454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:19:01.125454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:19:01.125454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:19:01.125454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 12:19:01.125454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:19:01.125454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:19:01.125454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:19:01.125454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:01.125454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:01.125454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:01.125454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:01.125454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:01.126459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:01.126459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:01.126459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:01.126459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:01.126459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:01.126459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:01.126459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:19:01.126459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:19:01.126459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:19:01.126459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:19:01.126459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:01.126459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:01.126459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:01.126459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:01.126459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:01.126459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:01.127393 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:01.127393 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:01.127393 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:01.127393 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:01.127393 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:01.127393 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:01.127393 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:01.127393 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:01.127393 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:01.127393 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:01.127393 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:19:01.127393 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:19:01.127393 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 06 12:19:01.127393 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:19:01.127393 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 497/1002] 10.222.0.20 () {56 vars in 1126 bytes} [Sat Aug 6 12:19:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:19:01.133972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43901e-cbaa-48fc-96b2-080a99cf2c2d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:01.141451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4dcb42-0753-4f10-988e-0c3d8620fe64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:01.142080 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4dcb42-0753-4f10-988e-0c3d8620fe64 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:01.142364 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4dcb42-0753-4f10-988e-0c3d8620fe64 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:01.142642 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4dcb42-0753-4f10-988e-0c3d8620fe64 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:01.143406 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4dcb42-0753-4f10-988e-0c3d8620fe64 None admin] RBAC: Adding query filter params (interface=public, service_id=c3286345778246f8a58882f2c4b8717b, region_id=RegionOne) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:01.143717 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4dcb42-0753-4f10-988e-0c3d8620fe64 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:01.145125 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4dcb42-0753-4f10-988e-0c3d8620fe64 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:01.151523 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 506/1003] 10.222.0.20 () {56 vars in 1251 bytes} [Sat Aug 6 12:19:00 2022] GET /identity/v3/endpoints?service_id=c3286345778246f8a58882f2c4b8717b&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:01.157189 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4dcb42-0753-4f10-988e-0c3d8620fe64 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:01.187956 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:01.253563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43901e-cbaa-48fc-96b2-080a99cf2c2d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:01.266687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4dcb42-0753-4f10-988e-0c3d8620fe64 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:01.287813 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:01.296992 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd4c61-261c-4d57-be41-c5dc9bc2dd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:01.297538 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd4c61-261c-4d57-be41-c5dc9bc2dd84 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:01.297823 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd4c61-261c-4d57-be41-c5dc9bc2dd84 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:01.298095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd4c61-261c-4d57-be41-c5dc9bc2dd84 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:01.299155 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd4c61-261c-4d57-be41-c5dc9bc2dd84 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:01.299709 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd4c61-261c-4d57-be41-c5dc9bc2dd84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:01.301841 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd4c61-261c-4d57-be41-c5dc9bc2dd84 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:01.308331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 498/1004] 10.222.0.20 () {56 vars in 1122 bytes} [Sat Aug 6 12:19:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:01.313731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd4c61-261c-4d57-be41-c5dc9bc2dd84 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:01.314656 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5ed2e-022f-4f8f-bf7f-b267bdcc143f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:01.315294 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5ed2e-022f-4f8f-bf7f-b267bdcc143f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:01.315574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5ed2e-022f-4f8f-bf7f-b267bdcc143f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:01.315847 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5ed2e-022f-4f8f-bf7f-b267bdcc143f None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:01.316554 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5ed2e-022f-4f8f-bf7f-b267bdcc143f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:01.318470 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5ed2e-022f-4f8f-bf7f-b267bdcc143f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:01.325074 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 507/1005] 10.222.0.20 () {56 vars in 1087 bytes} [Sat Aug 6 12:19:01 2022] GET /identity/v3/services => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 12:19:01.428314 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd4c61-261c-4d57-be41-c5dc9bc2dd84 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:01.437940 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:01.469305 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97fabb-dd5a-42b0-9d4d-0edd95234931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:01.469936 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97fabb-dd5a-42b0-9d4d-0edd95234931 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:01.470213 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97fabb-dd5a-42b0-9d4d-0edd95234931 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:01.470484 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97fabb-dd5a-42b0-9d4d-0edd95234931 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:01.471168 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97fabb-dd5a-42b0-9d4d-0edd95234931 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:01.471469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97fabb-dd5a-42b0-9d4d-0edd95234931 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:01.472879 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97fabb-dd5a-42b0-9d4d-0edd95234931 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:01.483544 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 499/1006] 10.222.0.20 () {56 vars in 1122 bytes} [Sat Aug 6 12:19:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:01.484863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5ed2e-022f-4f8f-bf7f-b267bdcc143f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:01.594204 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5ed2e-022f-4f8f-bf7f-b267bdcc143f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:01.627645 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b91766-ef11-4c9b-a4e4-e5d665c5ea26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:01.628256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b91766-ef11-4c9b-a4e4-e5d665c5ea26 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:01.628579 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b91766-ef11-4c9b-a4e4-e5d665c5ea26 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:01.628971 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b91766-ef11-4c9b-a4e4-e5d665c5ea26 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:01.629557 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b91766-ef11-4c9b-a4e4-e5d665c5ea26 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:01.630984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b91766-ef11-4c9b-a4e4-e5d665c5ea26 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:01.649236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 508/1007] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:19:01 2022] POST /identity/v3/endpoints => generated 325 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:19:02.604628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afa8e1-7397-4972-9df1-fd5ed9839d83 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:02.605178 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afa8e1-7397-4972-9df1-fd5ed9839d83 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:02.605633 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afa8e1-7397-4972-9df1-fd5ed9839d83 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:02.607261 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 500/1008] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:19:02.613401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2328d-b8d3-4eba-b293-eb2f9151dfb0 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:02.613816 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2328d-b8d3-4eba-b293-eb2f9151dfb0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:02.614060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f2328d-b8d3-4eba-b293-eb2f9151dfb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:02.635547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e3f2328d-b8d3-4eba-b293-eb2f9151dfb0 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:02.636756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e3f2328d-b8d3-4eba-b293-eb2f9151dfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:02.670409 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:02.711378 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 509/1009] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:02 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:19:02.733476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afa8e1-7397-4972-9df1-fd5ed9839d83 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:02.838430 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afa8e1-7397-4972-9df1-fd5ed9839d83 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:02.871325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77d6d9-6ffc-4fdc-bee6-2461999596a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:02.871857 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77d6d9-6ffc-4fdc-bee6-2461999596a3 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:02.872134 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77d6d9-6ffc-4fdc-bee6-2461999596a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:02.872432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77d6d9-6ffc-4fdc-bee6-2461999596a3 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:02.873128 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77d6d9-6ffc-4fdc-bee6-2461999596a3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:02.874477 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77d6d9-6ffc-4fdc-bee6-2461999596a3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:02.896345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 501/1010] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Aug 6 12:19:02 2022] POST /identity/v3/services => generated 242 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:19:03.380803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92235a60-3e3d-49fb-8e70-0e5c30f3a911 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:03.381198 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92235a60-3e3d-49fb-8e70-0e5c30f3a911 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:03.381390 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92235a60-3e3d-49fb-8e70-0e5c30f3a911 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:03.383458 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 510/1011] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:19:03.388528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca17f31-1b7a-403d-b39d-f61a16f4f110 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:03.388820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca17f31-1b7a-403d-b39d-f61a16f4f110 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:03.389096 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca17f31-1b7a-403d-b39d-f61a16f4f110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:03.407078 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-5ca17f31-1b7a-403d-b39d-f61a16f4f110 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:03.408264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5ca17f31-1b7a-403d-b39d-f61a16f4f110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:03.435361 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:03.485143 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 502/1012] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:03 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:19:03.509170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92235a60-3e3d-49fb-8e70-0e5c30f3a911 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:03.614570 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92235a60-3e3d-49fb-8e70-0e5c30f3a911 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:03.650602 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74807864-2c39-4123-864d-917277e8d6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:03.651512 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74807864-2c39-4123-864d-917277e8d6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:03.651794 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74807864-2c39-4123-864d-917277e8d6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:03.652062 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74807864-2c39-4123-864d-917277e8d6c3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:03.652713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74807864-2c39-4123-864d-917277e8d6c3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:03.654070 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74807864-2c39-4123-864d-917277e8d6c3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:03.661774 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c11729-e929-4713-9ebf-71128206becd None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:03.662217 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c11729-e929-4713-9ebf-71128206becd None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:03.662308 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c11729-e929-4713-9ebf-71128206becd None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:03.663019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-74807864-2c39-4123-864d-917277e8d6c3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 12:19:03.663019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:03.663019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:19:03.663019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:19:03.663019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:19:03.663019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:19:03.663019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:19:03.663019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:19:03.663019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 12:19:03.663019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:19:03.663019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:19:03.663019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:19:03.663019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:03.663019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:03.663019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:03.663019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:03.663019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:03.669237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:03.669237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:03.669237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:03.669237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:03.669237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:03.669237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:03.669237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:19:03.669237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:19:03.669237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:19:03.669237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:19:03.669237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:03.669237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:03.669237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:03.669237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:03.669237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:03.669237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:03.670170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:03.670170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:03.670170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:03.670170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:03.670170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:03.670170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:03.670170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:03.670170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:03.670170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:03.670170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:03.670170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:19:03.670170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:19:03.670170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 06 12:19:03.670170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:19:03.670911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 503/1013] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:19:03.670982 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 511/1014] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:19:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:19:03.671063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d6960-b050-4eb7-9f21-85804e9050b8 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:03.671063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d6960-b050-4eb7-9f21-85804e9050b8 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:03.671063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d6960-b050-4eb7-9f21-85804e9050b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:03.672612 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74807864-2c39-4123-864d-917277e8d6c3 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:03.689154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-de4d6960-b050-4eb7-9f21-85804e9050b8 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:03.690626 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-de4d6960-b050-4eb7-9f21-85804e9050b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:03.705824 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:03.764086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 504/1015] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:03 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:19:03.779080 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74807864-2c39-4123-864d-917277e8d6c3 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:03.786689 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d6960-b050-4eb7-9f21-85804e9050b8 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:03.814722 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf573a98-6470-47d3-b084-22069ed879e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:03.815339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf573a98-6470-47d3-b084-22069ed879e8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:03.815621 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf573a98-6470-47d3-b084-22069ed879e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:03.815888 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf573a98-6470-47d3-b084-22069ed879e8 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:03.816596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf573a98-6470-47d3-b084-22069ed879e8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:03.816902 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf573a98-6470-47d3-b084-22069ed879e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:03.818281 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf573a98-6470-47d3-b084-22069ed879e8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:03.824246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 512/1016] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Aug 6 12:19:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:03.829976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf573a98-6470-47d3-b084-22069ed879e8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:03.880924 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:03.894931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d6960-b050-4eb7-9f21-85804e9050b8 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:03.933215 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:03.934980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dca05-b772-4a27-9b5d-357564a982f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:03.935957 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dca05-b772-4a27-9b5d-357564a982f1 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:03.936152 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dca05-b772-4a27-9b5d-357564a982f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:03.936466 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dca05-b772-4a27-9b5d-357564a982f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:03.937626 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804dca05-b772-4a27-9b5d-357564a982f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:03.939072 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804dca05-b772-4a27-9b5d-357564a982f1 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:03.940894 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 505/1017] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:19:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:03.941145 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf573a98-6470-47d3-b084-22069ed879e8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:03.946659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dca05-b772-4a27-9b5d-357564a982f1 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:03.980011 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea630a-4e84-4ff2-a456-1bf552f41c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:03.980679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea630a-4e84-4ff2-a456-1bf552f41c9d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:03.980956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea630a-4e84-4ff2-a456-1bf552f41c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:03.981223 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea630a-4e84-4ff2-a456-1bf552f41c9d None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:03.981892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea630a-4e84-4ff2-a456-1bf552f41c9d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:03.982211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea630a-4e84-4ff2-a456-1bf552f41c9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:03.983536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea630a-4e84-4ff2-a456-1bf552f41c9d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:03.988629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 513/1018] 10.222.0.20 () {56 vars in 1112 bytes} [Sat Aug 6 12:19:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:03.997918 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea630a-4e84-4ff2-a456-1bf552f41c9d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:04.062974 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dca05-b772-4a27-9b5d-357564a982f1 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:04.099477 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dcf3cc-b6a6-4b4d-a003-e9fc115f3612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:04.100092 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcf3cc-b6a6-4b4d-a003-e9fc115f3612 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:04.100402 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcf3cc-b6a6-4b4d-a003-e9fc115f3612 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:04.101210 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcf3cc-b6a6-4b4d-a003-e9fc115f3612 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:04.101917 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dcf3cc-b6a6-4b4d-a003-e9fc115f3612 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:04.103433 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dcf3cc-b6a6-4b4d-a003-e9fc115f3612 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:04.107376 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea630a-4e84-4ff2-a456-1bf552f41c9d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:04.132537 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 506/1019] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Aug 6 12:19:03 2022] POST /identity/v3/projects => generated 311 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 12:19:04.159658 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e2072-5e87-49c8-aac6-8012779fea2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:04.160402 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e2072-5e87-49c8-aac6-8012779fea2f None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:04.160725 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e2072-5e87-49c8-aac6-8012779fea2f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:04.161001 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e2072-5e87-49c8-aac6-8012779fea2f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:04.161768 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e2072-5e87-49c8-aac6-8012779fea2f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:04.163792 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95e2072-5e87-49c8-aac6-8012779fea2f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:04.186342 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 514/1020] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:19:03 2022] POST /identity/v3/endpoints => generated 334 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:19:04.902974 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253c9e6-c7eb-4616-9efb-c09ba674418b None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:04.903519 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253c9e6-c7eb-4616-9efb-c09ba674418b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:04.903519 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253c9e6-c7eb-4616-9efb-c09ba674418b None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:04.904929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 507/1021] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:19:04.909970 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75483eb-f12a-4a95-b571-e2139586f5c7 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:04.910368 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75483eb-f12a-4a95-b571-e2139586f5c7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:04.910527 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75483eb-f12a-4a95-b571-e2139586f5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:04.928488 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a75483eb-f12a-4a95-b571-e2139586f5c7 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:04.930191 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a75483eb-f12a-4a95-b571-e2139586f5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:04.939718 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:05.009662 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 515/1022] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:19:05.032563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253c9e6-c7eb-4616-9efb-c09ba674418b None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:05.044855 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:05.140571 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253c9e6-c7eb-4616-9efb-c09ba674418b None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:05.173524 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a00ea-d159-478c-b6a7-430affea0f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:05.174086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a00ea-d159-478c-b6a7-430affea0f69 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:05.174362 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a00ea-d159-478c-b6a7-430affea0f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:05.174636 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a00ea-d159-478c-b6a7-430affea0f69 None admin] PATH_INFO: `/v3/services/image` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:05.175258 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6a00ea-d159-478c-b6a7-430affea0f69 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:05.176677 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6a00ea-d159-478c-b6a7-430affea0f69 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:05.185902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-db6a00ea-d159-478c-b6a7-430affea0f69 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 12:19:05.185902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:05.185902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:19:05.185902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:19:05.185902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:19:05.185902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:19:05.185902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:19:05.185902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:19:05.185902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 12:19:05.185902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:19:05.185902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:19:05.185902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:19:05.185902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:05.185902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:05.185902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:05.185902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:05.185902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:05.186848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:05.186848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:05.186848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:05.186848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:05.186848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:05.186848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:05.186848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:19:05.186848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:19:05.186848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:19:05.186848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:19:05.186848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:05.186848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:05.186848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:05.186848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:05.186848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:05.186848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:05.187685 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:05.187685 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:05.187685 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:05.187685 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:05.187685 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:05.187685 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:05.187685 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:05.187685 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:05.187685 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:05.187685 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:05.187685 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:19:05.187685 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:19:05.187685 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 12:19:05.187685 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:19:05.187685 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 508/1023] 10.222.0.20 () {56 vars in 1105 bytes} [Sat Aug 6 12:19:05 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:19:05.195365 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75483eb-f12a-4a95-b571-e2139586f5c7 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:05.296108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75483eb-f12a-4a95-b571-e2139586f5c7 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:05.330240 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a337b-3deb-4c7e-a72c-292e78121428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:05.330859 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a337b-3deb-4c7e-a72c-292e78121428 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:05.331133 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a337b-3deb-4c7e-a72c-292e78121428 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:05.331399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a337b-3deb-4c7e-a72c-292e78121428 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:05.332069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a337b-3deb-4c7e-a72c-292e78121428 None admin] RBAC: Adding query filter params (name=image) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:05.332389 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a337b-3deb-4c7e-a72c-292e78121428 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:05.333741 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a337b-3deb-4c7e-a72c-292e78121428 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:05.340487 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 516/1024] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:19:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:05.346066 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a00ea-d159-478c-b6a7-430affea0f69 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:05.419823 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:05.450927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a00ea-d159-478c-b6a7-430affea0f69 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:05.483608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb989b-2e53-414e-9722-41cf8f744969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:05.484217 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb989b-2e53-414e-9722-41cf8f744969 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:05.484552 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb989b-2e53-414e-9722-41cf8f744969 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:05.484827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb989b-2e53-414e-9722-41cf8f744969 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:05.485502 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb989b-2e53-414e-9722-41cf8f744969 None admin] RBAC: Adding query filter params (type=image) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:05.485807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb989b-2e53-414e-9722-41cf8f744969 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:05.487163 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb989b-2e53-414e-9722-41cf8f744969 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:05.492858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 509/1025] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:19:05 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:05.498535 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a337b-3deb-4c7e-a72c-292e78121428 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:05.502295 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:05.597666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a337b-3deb-4c7e-a72c-292e78121428 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:05.634195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc7b9a-50da-42d6-913c-bb60a3652cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:05.634811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc7b9a-50da-42d6-913c-bb60a3652cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:05.635090 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc7b9a-50da-42d6-913c-bb60a3652cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:05.635362 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc7b9a-50da-42d6-913c-bb60a3652cf6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:05.636114 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc7b9a-50da-42d6-913c-bb60a3652cf6 None admin] RBAC: Adding query filter params (interface=public, service_id=766f6f8299fb462b8504676e1dfd9e6c, region_id=RegionOne) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:05.636451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc7b9a-50da-42d6-913c-bb60a3652cf6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:05.637807 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc7b9a-50da-42d6-913c-bb60a3652cf6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:05.647958 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 517/1026] 10.222.0.20 () {56 vars in 1251 bytes} [Sat Aug 6 12:19:05 2022] GET /identity/v3/endpoints?service_id=766f6f8299fb462b8504676e1dfd9e6c&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:05.649237 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb989b-2e53-414e-9722-41cf8f744969 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:05.747848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb989b-2e53-414e-9722-41cf8f744969 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:05.780905 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839bc22-65a5-402c-9c6a-9bf1e110ef08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:05.781524 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839bc22-65a5-402c-9c6a-9bf1e110ef08 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:05.781805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839bc22-65a5-402c-9c6a-9bf1e110ef08 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:05.782075 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839bc22-65a5-402c-9c6a-9bf1e110ef08 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:05.782715 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839bc22-65a5-402c-9c6a-9bf1e110ef08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:05.784091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839bc22-65a5-402c-9c6a-9bf1e110ef08 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:05.785894 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:05.800348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 510/1027] 10.222.0.20 () {56 vars in 1087 bytes} [Sat Aug 6 12:19:05 2022] GET /identity/v3/services => generated 1957 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:19:06.194009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6a6ea-bf45-4154-8cd6-2891cf2be18e None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:06.195043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6a6ea-bf45-4154-8cd6-2891cf2be18e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:06.195120 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6a6ea-bf45-4154-8cd6-2891cf2be18e None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:06.196600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 518/1028] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:19:06.201686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b99acb9-2dd4-48ea-9d14-d274709e7d6d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:06.202092 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b99acb9-2dd4-48ea-9d14-d274709e7d6d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:06.202261 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b99acb9-2dd4-48ea-9d14-d274709e7d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:06.221245 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-4b99acb9-2dd4-48ea-9d14-d274709e7d6d None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:06.222424 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4b99acb9-2dd4-48ea-9d14-d274709e7d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:06.291482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 511/1029] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:19:06.315069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6a6ea-bf45-4154-8cd6-2891cf2be18e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:06.356201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471aa690-1f17-4fd6-8865-599c2fe3929d None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:06.356516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471aa690-1f17-4fd6-8865-599c2fe3929d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:06.356797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471aa690-1f17-4fd6-8865-599c2fe3929d None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:06.358274 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 512/1030] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:19:06.362815 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be71baf-e690-4a44-91b7-2e6401cc76ef None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:06.363099 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be71baf-e690-4a44-91b7-2e6401cc76ef None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:06.363375 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be71baf-e690-4a44-91b7-2e6401cc76ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:06.383991 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3be71baf-e690-4a44-91b7-2e6401cc76ef None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:06.385385 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3be71baf-e690-4a44-91b7-2e6401cc76ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:06.386437 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:06.425731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6a6ea-bf45-4154-8cd6-2891cf2be18e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:06.457637 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 513/1031] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 12:19:06.466147 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51ecb2-3162-4fcd-8905-c4b40af59a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:06.466785 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51ecb2-3162-4fcd-8905-c4b40af59a4d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:06.467059 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51ecb2-3162-4fcd-8905-c4b40af59a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:06.467334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51ecb2-3162-4fcd-8905-c4b40af59a4d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:06.468557 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51ecb2-3162-4fcd-8905-c4b40af59a4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:06.469971 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51ecb2-3162-4fcd-8905-c4b40af59a4d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:06.476379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 519/1032] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:19:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:06.477406 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be71baf-e690-4a44-91b7-2e6401cc76ef None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:06.480992 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51ecb2-3162-4fcd-8905-c4b40af59a4d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:06.541396 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:06.582121 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be71baf-e690-4a44-91b7-2e6401cc76ef None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:06.585998 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51ecb2-3162-4fcd-8905-c4b40af59a4d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:06.617773 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ce036-aa0e-4044-9746-7855c8f89827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:06.618384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ce036-aa0e-4044-9746-7855c8f89827 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:06.618662 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ce036-aa0e-4044-9746-7855c8f89827 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:06.618935 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ce036-aa0e-4044-9746-7855c8f89827 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:06.619633 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ce036-aa0e-4044-9746-7855c8f89827 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:06.621057 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ce036-aa0e-4044-9746-7855c8f89827 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:06.621560 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a37bdd-5995-4de1-ab91-94ed6667ebdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:06.622197 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a37bdd-5995-4de1-ab91-94ed6667ebdd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:06.622481 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a37bdd-5995-4de1-ab91-94ed6667ebdd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:06.622754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a37bdd-5995-4de1-ab91-94ed6667ebdd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:06.625740 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:06.637665 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a37bdd-5995-4de1-ab91-94ed6667ebdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:19:06.642710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 514/1033] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Aug 6 12:19:06 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:19:06.642828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-85a37bdd-5995-4de1-ab91-94ed6667ebdd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 12:19:06.642828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:06.642828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:06.642828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:06.642828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:06.642828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:06.642828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:06.642828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:06.642828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:06.642828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:06.642828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:06.642828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:06.642828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:19:06.642828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:19:06.642828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:19:06.642828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:19:06.644157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:19:06.644157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:19:06.644157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:19:06.644157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:19:06.644157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:06.644157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:06.644157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:06.644157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:06.644157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:06.644157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:06.644157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:06.644157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:06.644157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:06.644157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:06.644157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:06.644157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:06.645333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:06.645333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:06.645333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:06.645333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:06.645333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:19:06.645333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:19:06.645333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:19:06.645333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:19:06.645333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:19:06.645333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:19:06.645333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 12:19:06.645333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:19:06.649697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 520/1034] 10.222.0.20 () {56 vars in 1120 bytes} [Sat Aug 6 12:19:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:19:06.653499 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ce036-aa0e-4044-9746-7855c8f89827 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:06.762925 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ce036-aa0e-4044-9746-7855c8f89827 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:06.780972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:06.804831 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8b6ea-5173-4af7-9cbc-2730546cb53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:06.805460 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8b6ea-5173-4af7-9cbc-2730546cb53f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:06.805739 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8b6ea-5173-4af7-9cbc-2730546cb53f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:06.806019 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8b6ea-5173-4af7-9cbc-2730546cb53f None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:06.806741 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f8b6ea-5173-4af7-9cbc-2730546cb53f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:06.807051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f8b6ea-5173-4af7-9cbc-2730546cb53f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:06.808488 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f8b6ea-5173-4af7-9cbc-2730546cb53f None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:06.816611 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 515/1035] 10.222.0.20 () {56 vars in 1115 bytes} [Sat Aug 6 12:19:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:06.824797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a37bdd-5995-4de1-ab91-94ed6667ebdd None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:06.937040 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a37bdd-5995-4de1-ab91-94ed6667ebdd None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:06.975171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b32167-b9ac-4610-aaec-8c9472c3e75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:06.976838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b32167-b9ac-4610-aaec-8c9472c3e75d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:06.977357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b32167-b9ac-4610-aaec-8c9472c3e75d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:06.977856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b32167-b9ac-4610-aaec-8c9472c3e75d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:06.990177 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b32167-b9ac-4610-aaec-8c9472c3e75d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:06.996419 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b32167-b9ac-4610-aaec-8c9472c3e75d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:07.013303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-43b32167-b9ac-4610-aaec-8c9472c3e75d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:19:07.013303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:07.013303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:07.013303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:07.013303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:07.013303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:07.013303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:07.013303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:07.013303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:07.013303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:07.013303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:07.013303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:07.013303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:19:07.013303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:19:07.013303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:19:07.014246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:19:07.014246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:07.014246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:07.014246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:07.014246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:07.014246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:07.014246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:07.014246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:07.014246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:07.014246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:07.014246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:07.014246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:07.014246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:07.014246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:07.014246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:07.014246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:07.014246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:07.015158 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:19:07.015158 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:19:07.015158 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:19:07.015158 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:19:07.015158 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 521/1036] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:19:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:19:07.021503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8b6ea-5173-4af7-9cbc-2730546cb53f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:07.123818 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8b6ea-5173-4af7-9cbc-2730546cb53f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:07.157129 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebae689-33b4-4bd5-bfe4-826e6047473c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:07.157749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebae689-33b4-4bd5-bfe4-826e6047473c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:07.158029 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebae689-33b4-4bd5-bfe4-826e6047473c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:07.158301 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebae689-33b4-4bd5-bfe4-826e6047473c None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:07.159213 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebae689-33b4-4bd5-bfe4-826e6047473c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:07.159595 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebae689-33b4-4bd5-bfe4-826e6047473c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:07.161283 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebae689-33b4-4bd5-bfe4-826e6047473c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:07.174723 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 516/1037] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:19:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:07.180251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b32167-b9ac-4610-aaec-8c9472c3e75d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:07.230431 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:07.283437 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b32167-b9ac-4610-aaec-8c9472c3e75d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:07.316923 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f6c55-064f-4ce7-9236-33bdfec03cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:07.317581 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f6c55-064f-4ce7-9236-33bdfec03cab None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:07.317862 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f6c55-064f-4ce7-9236-33bdfec03cab None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:07.318137 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f6c55-064f-4ce7-9236-33bdfec03cab None admin] PATH_INFO: `/v3/users/nova` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:07.327047 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f6c55-064f-4ce7-9236-33bdfec03cab None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:07.328518 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f6c55-064f-4ce7-9236-33bdfec03cab None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:07.340524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-8e3f6c55-064f-4ce7-9236-33bdfec03cab None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 12:19:07.340524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:07.340524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:07.340524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:07.340524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:07.340524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:07.340524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:07.340524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:07.340524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:07.340524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:07.340524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:07.340524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:07.340524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:19:07.340524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:19:07.340524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:19:07.340524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:19:07.341547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:07.341547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:07.341547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:19:07.341547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:19:07.341547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:19:07.341547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:19:07.341547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:07.341547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:07.341547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:07.341547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:07.341547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:07.341547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:07.341547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:07.341547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:07.341547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:07.341547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:07.341547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:07.341547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:07.342496 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:07.342496 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:07.342496 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:19:07.342496 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:19:07.342496 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:19:07.342496 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:19:07.342496 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:19:07.342496 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:19:07.342496 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 12:19:07.342496 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:19:07.343912 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 522/1038] 10.222.0.20 () {56 vars in 1093 bytes} [Sat Aug 6 12:19:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:19:07.347410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebae689-33b4-4bd5-bfe4-826e6047473c None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:07.390008 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:07.451474 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebae689-33b4-4bd5-bfe4-826e6047473c None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:07.486988 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13f619-4637-4702-8afa-bd568cb393a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:07.487523 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13f619-4637-4702-8afa-bd568cb393a4 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:07.487851 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13f619-4637-4702-8afa-bd568cb393a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:07.488130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13f619-4637-4702-8afa-bd568cb393a4 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:07.488944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13f619-4637-4702-8afa-bd568cb393a4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:07.489248 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13f619-4637-4702-8afa-bd568cb393a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:07.490732 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13f619-4637-4702-8afa-bd568cb393a4 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:07.513099 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 517/1039] 10.222.0.20 () {56 vars in 1133 bytes} [Sat Aug 6 12:19:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:07.514479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f6c55-064f-4ce7-9236-33bdfec03cab None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:07.620450 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f6c55-064f-4ce7-9236-33bdfec03cab None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:07.632533 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:07.659205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21478f5b-0a74-42fd-842d-7da1951d8e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:07.659823 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21478f5b-0a74-42fd-842d-7da1951d8e10 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:07.660101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21478f5b-0a74-42fd-842d-7da1951d8e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:07.660396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21478f5b-0a74-42fd-842d-7da1951d8e10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:07.729555 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21478f5b-0a74-42fd-842d-7da1951d8e10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:07.730996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21478f5b-0a74-42fd-842d-7da1951d8e10 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:07.802871 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd62db-da26-4042-81cc-6a5afa67932c None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:07.803227 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd62db-da26-4042-81cc-6a5afa67932c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:07.803427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd62db-da26-4042-81cc-6a5afa67932c None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:07.805460 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 518/1040] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:19:07.810870 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebf053-cede-4e80-8d09-bb3186dec2c9 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:07.811190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebf053-cede-4e80-8d09-bb3186dec2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:07.811418 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebf053-cede-4e80-8d09-bb3186dec2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:07.830182 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-10ebf053-cede-4e80-8d09-bb3186dec2c9 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:07.831347 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-10ebf053-cede-4e80-8d09-bb3186dec2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:07.842955 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 523/1041] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:19:07 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 06 12:19:07.843608 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21478f5b-0a74-42fd-842d-7da1951d8e10 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:07.854001 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:07.908780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 519/1042] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 12:19:07.931156 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ebf053-cede-4e80-8d09-bb3186dec2c9 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:07.945107 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21478f5b-0a74-42fd-842d-7da1951d8e10 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:07.956958 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:07.985063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d8630-0c8f-466a-a182-d4e41f49a27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:07.985720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d8630-0c8f-466a-a182-d4e41f49a27c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:07.986003 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d8630-0c8f-466a-a182-d4e41f49a27c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:07.986314 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d8630-0c8f-466a-a182-d4e41f49a27c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:07.998224 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d8630-0c8f-466a-a182-d4e41f49a27c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:07.999151 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d8630-0c8f-466a-a182-d4e41f49a27c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:08.014868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-bf9d8630-0c8f-466a-a182-d4e41f49a27c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:19:08.014868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:08.014868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:08.014868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:08.014868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:08.014868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:08.014868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:08.014868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:08.014868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:08.014868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:08.014868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:08.014868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:08.014868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:19:08.014868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:19:08.014868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:19:08.015743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:19:08.015743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:08.015743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:08.015743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:08.015743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:08.015743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:08.015743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:08.015743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:08.015743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:08.015743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:08.015743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:08.015743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:08.015743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:08.015743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:08.015743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:08.015743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:08.015743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:08.016692 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:19:08.016692 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:19:08.016692 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:19:08.016692 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:19:08.020652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 524/1043] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:19:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:19:08.021436 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d8630-0c8f-466a-a182-d4e41f49a27c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:08.044712 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ebf053-cede-4e80-8d09-bb3186dec2c9 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:08.079904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4f074-03e1-406e-aa5f-585b695a2db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:08.080505 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4f074-03e1-406e-aa5f-585b695a2db9 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:08.080785 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4f074-03e1-406e-aa5f-585b695a2db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:08.081062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca4f074-03e1-406e-aa5f-585b695a2db9 None admin] PATH_INFO: `/v3/services/image` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:08.081690 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4f074-03e1-406e-aa5f-585b695a2db9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:08.083049 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca4f074-03e1-406e-aa5f-585b695a2db9 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:08.086924 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:08.093014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-dca4f074-03e1-406e-aa5f-585b695a2db9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 12:19:08.093014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:08.093014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:19:08.093014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:19:08.093014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:19:08.093014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:19:08.093014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:19:08.093014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:19:08.093014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 12:19:08.093014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:19:08.093014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:19:08.093014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:19:08.093014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:08.093014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:08.093014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:08.093014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:08.093014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:08.093999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:08.093999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:08.093999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:08.093999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:08.093999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:08.093999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:08.093999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:19:08.093999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:19:08.093999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:19:08.093999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:19:08.093999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:08.093999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:08.093999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:08.093999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:08.093999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:08.093999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:08.094827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:08.094827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:08.094827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:08.094827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:08.094827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:08.094827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:08.094827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:08.094827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:08.094827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:08.094827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:08.094827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:19:08.094827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:19:08.094827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 06 12:19:08.094827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:19:08.099180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 520/1044] 10.222.0.20 () {56 vars in 1105 bytes} [Sat Aug 6 12:19:07 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:19:08.100044 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4f074-03e1-406e-aa5f-585b695a2db9 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:08.133755 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d8630-0c8f-466a-a182-d4e41f49a27c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:08.169172 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a94033-1527-4f24-bda0-fd10fa7331e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:08.169791 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a94033-1527-4f24-bda0-fd10fa7331e7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:08.170070 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a94033-1527-4f24-bda0-fd10fa7331e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:08.170341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a94033-1527-4f24-bda0-fd10fa7331e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:08.171026 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a94033-1527-4f24-bda0-fd10fa7331e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:08.171326 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a94033-1527-4f24-bda0-fd10fa7331e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:08.172915 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a94033-1527-4f24-bda0-fd10fa7331e7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:08.186292 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 525/1045] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:08.191125 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:08.191366 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a94033-1527-4f24-bda0-fd10fa7331e7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:08.204999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca4f074-03e1-406e-aa5f-585b695a2db9 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:08.238353 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be498f5-b072-4ae5-a7fe-10376343bd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:08.238969 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be498f5-b072-4ae5-a7fe-10376343bd05 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:08.239245 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be498f5-b072-4ae5-a7fe-10376343bd05 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:08.239512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be498f5-b072-4ae5-a7fe-10376343bd05 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:08.240181 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be498f5-b072-4ae5-a7fe-10376343bd05 None admin] RBAC: Adding query filter params (name=image) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:08.240529 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be498f5-b072-4ae5-a7fe-10376343bd05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:08.242651 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be498f5-b072-4ae5-a7fe-10376343bd05 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:08.250124 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 521/1046] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:19:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:08.255883 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be498f5-b072-4ae5-a7fe-10376343bd05 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:08.297971 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a94033-1527-4f24-bda0-fd10fa7331e7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:08.318401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:08.333115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44743176-7f91-45ab-839c-c5c08bf873f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:08.333680 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44743176-7f91-45ab-839c-c5c08bf873f1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:08.333955 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44743176-7f91-45ab-839c-c5c08bf873f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:08.334224 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44743176-7f91-45ab-839c-c5c08bf873f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:08.341433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44743176-7f91-45ab-839c-c5c08bf873f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:08.342970 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44743176-7f91-45ab-839c-c5c08bf873f1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:08.355060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-44743176-7f91-45ab-839c-c5c08bf873f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:19:08.355060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:08.355060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:08.355060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:08.355060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:08.355060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:08.355060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:08.355060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:08.355060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:08.355060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:08.355060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:08.355060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:08.355060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:19:08.355060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:19:08.355060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:19:08.355993 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:19:08.355993 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:08.355993 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:08.355993 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:08.355993 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:08.355993 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:08.355993 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:08.355993 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:08.355993 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:08.355993 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:08.355993 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:08.355993 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:08.355993 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:08.355993 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:08.355993 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:08.355993 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:08.355993 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:08.357128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:19:08.357128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:19:08.357128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:19:08.357128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:19:08.357128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:19:08.357128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:19:08.357128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:19:08.357128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:19:08.357128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 526/1047] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:19:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:19:08.361821 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44743176-7f91-45ab-839c-c5c08bf873f1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:08.363878 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be498f5-b072-4ae5-a7fe-10376343bd05 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:08.400526 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1169291-19bc-4f2e-8120-3b564427d965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:08.401136 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1169291-19bc-4f2e-8120-3b564427d965 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:08.401539 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1169291-19bc-4f2e-8120-3b564427d965 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:08.401814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1169291-19bc-4f2e-8120-3b564427d965 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:08.402491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1169291-19bc-4f2e-8120-3b564427d965 None admin] RBAC: Adding query filter params (type=image) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:08.402791 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1169291-19bc-4f2e-8120-3b564427d965 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:08.404282 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1169291-19bc-4f2e-8120-3b564427d965 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:08.409734 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 522/1048] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:19:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:08.415637 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1169291-19bc-4f2e-8120-3b564427d965 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:08.434264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:08.464106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44743176-7f91-45ab-839c-c5c08bf873f1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:08.500884 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad3cc9-4e9f-4dc8-95f9-28de796056da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:08.501504 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad3cc9-4e9f-4dc8-95f9-28de796056da None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:08.501779 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad3cc9-4e9f-4dc8-95f9-28de796056da None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:08.502087 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad3cc9-4e9f-4dc8-95f9-28de796056da None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:08.503082 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad3cc9-4e9f-4dc8-95f9-28de796056da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:08.503082 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad3cc9-4e9f-4dc8-95f9-28de796056da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:08.504505 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad3cc9-4e9f-4dc8-95f9-28de796056da None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:08.517762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 527/1049] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:19:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:08.523347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad3cc9-4e9f-4dc8-95f9-28de796056da None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:08.525598 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1169291-19bc-4f2e-8120-3b564427d965 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:08.563778 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48593fd7-ed72-4f30-89ea-0f085f804c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:08.564440 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48593fd7-ed72-4f30-89ea-0f085f804c99 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:08.564723 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48593fd7-ed72-4f30-89ea-0f085f804c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:08.564994 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48593fd7-ed72-4f30-89ea-0f085f804c99 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:08.566115 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48593fd7-ed72-4f30-89ea-0f085f804c99 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:08.567304 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:08.567853 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48593fd7-ed72-4f30-89ea-0f085f804c99 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:08.594483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 523/1050] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:19:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 12:19:08.633954 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad3cc9-4e9f-4dc8-95f9-28de796056da None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:08.679479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9abe9e-a921-4a28-a906-954b3081da53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:08.680194 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9abe9e-a921-4a28-a906-954b3081da53 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:08.680562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9abe9e-a921-4a28-a906-954b3081da53 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:08.680892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9abe9e-a921-4a28-a906-954b3081da53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:08.681750 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9abe9e-a921-4a28-a906-954b3081da53 None admin] RBAC: Adding query filter params (role.id=ed864363c041447ab55d1795a8c7092b, scope.project.id=13fba33c46cf40bbb15a08c690b4c98c, user.id=524b728bf63c4ca88e0f25e531cf5056) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:08.682125 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9abe9e-a921-4a28-a906-954b3081da53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:08.683619 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9abe9e-a921-4a28-a906-954b3081da53 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:08.696326 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 528/1051] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:19:08 2022] GET /identity/v3/role_assignments?user.id=524b728bf63c4ca88e0f25e531cf5056&scope.project.id=13fba33c46cf40bbb15a08c690b4c98c&role.id=ed864363c041447ab55d1795a8c7092b => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:08.731701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33e7ba-36f5-4bba-a6f7-32276e5e8ed5 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:08.732082 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33e7ba-36f5-4bba-a6f7-32276e5e8ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:08.732492 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33e7ba-36f5-4bba-a6f7-32276e5e8ed5 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:08.734032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 524/1052] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:19:08.740141 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1dc83-7467-43e0-af2f-82e8fda2e73b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:08.740481 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1dc83-7467-43e0-af2f-82e8fda2e73b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:08.740747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1dc83-7467-43e0-af2f-82e8fda2e73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:08.761806 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3af1dc83-7467-43e0-af2f-82e8fda2e73b None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:08.763056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3af1dc83-7467-43e0-af2f-82e8fda2e73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:08.841741 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 529/1053] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:19:08.865751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33e7ba-36f5-4bba-a6f7-32276e5e8ed5 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:08.944086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:08.972109 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33e7ba-36f5-4bba-a6f7-32276e5e8ed5 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:09.016572 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6f956-f903-4377-b1bd-75acc34f5bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:09.017214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6f956-f903-4377-b1bd-75acc34f5bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:09.017494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6f956-f903-4377-b1bd-75acc34f5bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:09.017770 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6f956-f903-4377-b1bd-75acc34f5bd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:09.029866 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6f956-f903-4377-b1bd-75acc34f5bd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:19:09.034838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-d0b6f956-f903-4377-b1bd-75acc34f5bd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:19:09.034838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:09.034838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:09.034838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:09.034838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:09.034838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:09.034838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:09.034838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:09.034838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:09.034838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:09.034838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:09.034838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:09.034838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:19:09.034838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:19:09.034838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:19:09.034838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:19:09.035769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:19:09.035769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:19:09.035769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:19:09.035769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:19:09.035769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:09.035769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:09.035769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:09.035769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:09.035769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:09.035769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:09.035769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:09.035769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:09.035769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:09.035769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:09.035769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:09.035769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:09.041005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:09.041005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:09.041005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:09.041005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:09.041005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:19:09.041005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:19:09.041005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:19:09.041005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:19:09.041005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:19:09.041005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:19:09.041005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:19:09.041005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:19:09.041005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 525/1054] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:19:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:19:09.044337 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1dc83-7467-43e0-af2f-82e8fda2e73b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:09.075333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:09.153535 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1dc83-7467-43e0-af2f-82e8fda2e73b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:09.187473 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ef050-281d-4b0a-b95a-471d2fcfbc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:09.188097 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ef050-281d-4b0a-b95a-471d2fcfbc96 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:09.188408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ef050-281d-4b0a-b95a-471d2fcfbc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:09.188687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ef050-281d-4b0a-b95a-471d2fcfbc96 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:09.189397 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ef050-281d-4b0a-b95a-471d2fcfbc96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:09.189698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ef050-281d-4b0a-b95a-471d2fcfbc96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:09.191066 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ef050-281d-4b0a-b95a-471d2fcfbc96 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:09.200116 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 530/1055] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:19:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:09.205684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6f956-f903-4377-b1bd-75acc34f5bd6 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:09.310650 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6f956-f903-4377-b1bd-75acc34f5bd6 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:09.350797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d66de-6fe9-4be5-90fd-3530d6a4e641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:09.351431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d66de-6fe9-4be5-90fd-3530d6a4e641 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:09.351714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d66de-6fe9-4be5-90fd-3530d6a4e641 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:09.351991 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d66de-6fe9-4be5-90fd-3530d6a4e641 None admin] PATH_INFO: `/v3/users/0459782d36d842f58a2f95f6167a229f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:09.354115 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:09.370636 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d66de-6fe9-4be5-90fd-3530d6a4e641 None admin] RBAC: Authorizing `identity:get_user(user_id=0459782d36d842f58a2f95f6167a229f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:09.372610 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d66de-6fe9-4be5-90fd-3530d6a4e641 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:09.376654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 526/1056] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:19:09 2022] GET /identity/v3/users/0459782d36d842f58a2f95f6167a229f => generated 285 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:09.381112 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ef050-281d-4b0a-b95a-471d2fcfbc96 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:09.473669 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:09.483926 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ef050-281d-4b0a-b95a-471d2fcfbc96 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:09.517191 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdcd31-5ac0-4d37-855f-5e60b5b59370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:09.517917 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdcd31-5ac0-4d37-855f-5e60b5b59370 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:09.518301 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdcd31-5ac0-4d37-855f-5e60b5b59370 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:09.518677 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdcd31-5ac0-4d37-855f-5e60b5b59370 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:09.586418 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabdcd31-5ac0-4d37-855f-5e60b5b59370 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:09.588017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabdcd31-5ac0-4d37-855f-5e60b5b59370 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:09.691812 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:09.707032 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 531/1057] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:19:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 12:19:09.708278 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d66de-6fe9-4be5-90fd-3530d6a4e641 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:09.814686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d66de-6fe9-4be5-90fd-3530d6a4e641 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:09.849700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcca7f6-a433-4ec5-8528-350a3c8e1e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:09.850496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcca7f6-a433-4ec5-8528-350a3c8e1e8d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:09.850877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcca7f6-a433-4ec5-8528-350a3c8e1e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:09.851251 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcca7f6-a433-4ec5-8528-350a3c8e1e8d None admin] PATH_INFO: `/v3/projects/db2b557448f04e4aabfb2859cd1be4a1` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:09.852550 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcca7f6-a433-4ec5-8528-350a3c8e1e8d None admin] RBAC: Authorizing `identity:get_project(project_id=db2b557448f04e4aabfb2859cd1be4a1)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:09.854079 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcca7f6-a433-4ec5-8528-350a3c8e1e8d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:09.856461 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 527/1058] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:19:09 2022] GET /identity/v3/projects/db2b557448f04e4aabfb2859cd1be4a1 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:09.862150 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdcd31-5ac0-4d37-855f-5e60b5b59370 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:09.961551 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdcd31-5ac0-4d37-855f-5e60b5b59370 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:10.008713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893516b6-1a32-4bed-a7f0-5ea4c2982497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:10.009598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893516b6-1a32-4bed-a7f0-5ea4c2982497 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:10.010083 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893516b6-1a32-4bed-a7f0-5ea4c2982497 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:10.010604 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893516b6-1a32-4bed-a7f0-5ea4c2982497 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:10.011669 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893516b6-1a32-4bed-a7f0-5ea4c2982497 None admin] RBAC: Adding query filter params (role.id=144c8425688b409384e1e8ed8279d88e, scope.project.id=db2b557448f04e4aabfb2859cd1be4a1, user.id=0459782d36d842f58a2f95f6167a229f) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:10.012185 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893516b6-1a32-4bed-a7f0-5ea4c2982497 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:10.013823 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893516b6-1a32-4bed-a7f0-5ea4c2982497 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:10.024334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 532/1059] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:19:09 2022] GET /identity/v3/role_assignments?user.id=0459782d36d842f58a2f95f6167a229f&scope.project.id=db2b557448f04e4aabfb2859cd1be4a1&role.id=144c8425688b409384e1e8ed8279d88e => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:10.682004 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49b684-16eb-4b4a-88fb-68ce6a4e30cd None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:10.682554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49b684-16eb-4b4a-88fb-68ce6a4e30cd None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:10.682554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49b684-16eb-4b4a-88fb-68ce6a4e30cd None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:10.683921 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 528/1060] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:19:10.688964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaed9a2-3b03-4c4a-98e5-fde3afb23139 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:10.689356 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaed9a2-3b03-4c4a-98e5-fde3afb23139 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:10.689545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaed9a2-3b03-4c4a-98e5-fde3afb23139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:10.696379 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68cade-6c77-405b-926d-a28e7e916e45 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:10.696669 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68cade-6c77-405b-926d-a28e7e916e45 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:10.696938 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68cade-6c77-405b-926d-a28e7e916e45 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:10.698400 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 529/1061] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:19:10.703257 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe624b-ee06-4151-89e6-f4a3484224bb None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:10.703754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe624b-ee06-4151-89e6-f4a3484224bb None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:10.704206 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe624b-ee06-4151-89e6-f4a3484224bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:10.707922 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-feaed9a2-3b03-4c4a-98e5-fde3afb23139 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:10.709131 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-feaed9a2-3b03-4c4a-98e5-fde3afb23139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:10.723538 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-97fe624b-ee06-4151-89e6-f4a3484224bb None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:10.725032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-97fe624b-ee06-4151-89e6-f4a3484224bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:10.763820 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:10.781984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 533/1062] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:19:10.782954 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:10.795363 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 530/1063] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 12:19:10.804682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaed9a2-3b03-4c4a-98e5-fde3afb23139 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:10.816913 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe624b-ee06-4151-89e6-f4a3484224bb None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:10.914810 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaed9a2-3b03-4c4a-98e5-fde3afb23139 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:10.924570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe624b-ee06-4151-89e6-f4a3484224bb None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:10.955861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016108de-e3b8-470e-85a1-282552d017c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:10.956595 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016108de-e3b8-470e-85a1-282552d017c3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:10.956846 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016108de-e3b8-470e-85a1-282552d017c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:10.957124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016108de-e3b8-470e-85a1-282552d017c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:10.959529 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a528a-8370-41bf-8710-a606e678d2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:10.960714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a528a-8370-41bf-8710-a606e678d2ab None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:10.960992 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a528a-8370-41bf-8710-a606e678d2ab None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:10.961272 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a528a-8370-41bf-8710-a606e678d2ab None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:10.971673 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a528a-8370-41bf-8710-a606e678d2ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:19:10.977148 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-ef5a528a-8370-41bf-8710-a606e678d2ab None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 12:19:10.977148 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:10.977148 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:10.977148 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:10.977148 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:10.977148 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:10.977148 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:10.977148 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:10.977148 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:10.977148 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:10.977148 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:10.977148 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:10.977148 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:19:10.977148 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:19:10.977148 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:19:10.977148 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:19:10.978135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:19:10.978135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:19:10.978135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:19:10.978135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:19:10.978135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:10.978135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:10.978135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:10.978135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:10.978135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:10.978135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:10.978135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:10.978135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:10.978135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:10.978135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:10.978135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:10.978135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:10.979026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:10.979026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:10.979026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:10.979026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:10.979026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:19:10.979026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:19:10.979026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:19:10.979026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:19:10.979026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:19:10.979026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:19:10.979026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 12:19:10.979026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:19:10.979026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 531/1064] 10.222.0.20 () {56 vars in 1120 bytes} [Sat Aug 6 12:19:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:19:10.985487 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a528a-8370-41bf-8710-a606e678d2ab None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:11.024999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:11.039057 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:11.040505 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016108de-e3b8-470e-85a1-282552d017c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:11.041989 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016108de-e3b8-470e-85a1-282552d017c3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:11.095122 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a528a-8370-41bf-8710-a606e678d2ab None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:11.132213 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b9c85-db95-46e3-9565-eef12ab88f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:11.132854 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b9c85-db95-46e3-9565-eef12ab88f0a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:11.133139 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b9c85-db95-46e3-9565-eef12ab88f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:11.133412 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b9c85-db95-46e3-9565-eef12ab88f0a None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:11.134126 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b9c85-db95-46e3-9565-eef12ab88f0a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:11.134430 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b9c85-db95-46e3-9565-eef12ab88f0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:11.135804 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b9c85-db95-46e3-9565-eef12ab88f0a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:11.144149 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 532/1065] 10.222.0.20 () {56 vars in 1115 bytes} [Sat Aug 6 12:19:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:11.153926 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 534/1066] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:19:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 12:19:11.154102 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b9c85-db95-46e3-9565-eef12ab88f0a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:11.154970 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016108de-e3b8-470e-85a1-282552d017c3 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:11.242250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:11.251743 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b9c85-db95-46e3-9565-eef12ab88f0a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:11.259047 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016108de-e3b8-470e-85a1-282552d017c3 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:11.265104 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:11.287558 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c4b3d6-d9ff-4192-ba80-1f8bade219a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:11.288755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4b3d6-d9ff-4192-ba80-1f8bade219a2 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:11.289032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4b3d6-d9ff-4192-ba80-1f8bade219a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:11.289310 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4b3d6-d9ff-4192-ba80-1f8bade219a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:11.296106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434a945-7ba9-40ff-8308-3fe3bead8ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:11.296770 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434a945-7ba9-40ff-8308-3fe3bead8ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:11.297055 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434a945-7ba9-40ff-8308-3fe3bead8ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:11.297330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434a945-7ba9-40ff-8308-3fe3bead8ac2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:11.299764 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c4b3d6-d9ff-4192-ba80-1f8bade219a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:11.301233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c4b3d6-d9ff-4192-ba80-1f8bade219a2 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:11.308052 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434a945-7ba9-40ff-8308-3fe3bead8ac2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:11.309519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434a945-7ba9-40ff-8308-3fe3bead8ac2 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:11.316033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-94c4b3d6-d9ff-4192-ba80-1f8bade219a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:19:11.316033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:11.316033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:11.316033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:11.316033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:11.316033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:11.316033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:11.316033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:11.316033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:11.316033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:11.316033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:11.316033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:11.316033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:19:11.316033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:19:11.316033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:19:11.317208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:19:11.317208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:11.317208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:11.317208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:11.317208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:11.317208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:11.317208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:11.317208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:11.317208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:11.317208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:11.317208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:11.317208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:11.317208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:11.317208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:11.317208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:11.317208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:11.317208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:11.318100 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:19:11.318100 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:19:11.318100 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:19:11.318100 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:19:11.318100 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 533/1067] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:19:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:19:11.322914 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c4b3d6-d9ff-4192-ba80-1f8bade219a2 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:11.327679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-f434a945-7ba9-40ff-8308-3fe3bead8ac2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:19:11.327679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:11.327679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:11.327679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:11.327679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:11.327679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:11.327679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:11.327679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:11.327679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:11.327679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:11.327679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:11.327679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:11.327679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:19:11.327679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:19:11.327679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:19:11.328798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:19:11.328798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:11.328798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:11.328798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:11.328798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:11.328798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:11.328798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:11.328798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:11.328798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:11.328798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:11.328798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:11.328798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:11.328798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:11.328798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:11.328798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:11.328798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:11.328798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:11.334803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:19:11.334803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:19:11.334803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:19:11.334803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:19:11.334803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 535/1068] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:19:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:19:11.334803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434a945-7ba9-40ff-8308-3fe3bead8ac2 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:11.426905 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c4b3d6-d9ff-4192-ba80-1f8bade219a2 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:11.445081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434a945-7ba9-40ff-8308-3fe3bead8ac2 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:11.465892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5485c81-106f-4ba1-910d-34870ae9bea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:11.467437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5485c81-106f-4ba1-910d-34870ae9bea7 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:11.469057 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5485c81-106f-4ba1-910d-34870ae9bea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:11.469057 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5485c81-106f-4ba1-910d-34870ae9bea7 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:11.469057 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5485c81-106f-4ba1-910d-34870ae9bea7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:11.469057 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5485c81-106f-4ba1-910d-34870ae9bea7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:11.472136 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5485c81-106f-4ba1-910d-34870ae9bea7 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:11.483100 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd864a8-cb4c-4481-8cf5-b46125f9958b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:11.483846 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd864a8-cb4c-4481-8cf5-b46125f9958b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:11.484225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd864a8-cb4c-4481-8cf5-b46125f9958b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:11.484689 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd864a8-cb4c-4481-8cf5-b46125f9958b None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:11.485528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 534/1069] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:11.489928 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd864a8-cb4c-4481-8cf5-b46125f9958b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:11.490270 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd864a8-cb4c-4481-8cf5-b46125f9958b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:11.491936 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5485c81-106f-4ba1-910d-34870ae9bea7 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:11.492538 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd864a8-cb4c-4481-8cf5-b46125f9958b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:11.499649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:11.505911 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 536/1070] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:11.523441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:11.607542 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5485c81-106f-4ba1-910d-34870ae9bea7 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:11.645407 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adedcbb-9c7b-4f7c-924f-2b34df90c205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:11.646159 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adedcbb-9c7b-4f7c-924f-2b34df90c205 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:11.646506 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adedcbb-9c7b-4f7c-924f-2b34df90c205 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:11.646854 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adedcbb-9c7b-4f7c-924f-2b34df90c205 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:11.655622 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adedcbb-9c7b-4f7c-924f-2b34df90c205 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:11.657121 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adedcbb-9c7b-4f7c-924f-2b34df90c205 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:11.672035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-4adedcbb-9c7b-4f7c-924f-2b34df90c205 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 12:19:11.672035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:11.672035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:11.672035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:11.672035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:11.672035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:11.672035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:11.672035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:11.672035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:11.672035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:11.672035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:11.672035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:11.672035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:19:11.672035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:19:11.672035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:19:11.672035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:19:11.673205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:11.673205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:11.673205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:19:11.673205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:19:11.673205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:19:11.673205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:19:11.673205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:11.673205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:11.673205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:11.673205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:11.673205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:11.673205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:11.673205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:11.673205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:11.673205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:11.673205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:11.673205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:11.673205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:11.674386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:11.674386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:11.674386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:19:11.674386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:19:11.674386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:19:11.674386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:19:11.674386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:19:11.674386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:19:11.674386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 12:19:11.674386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:19:11.674386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 535/1071] 10.222.0.20 () {56 vars in 1093 bytes} [Sat Aug 6 12:19:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:19:11.680205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd864a8-cb4c-4481-8cf5-b46125f9958b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:11.788849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd864a8-cb4c-4481-8cf5-b46125f9958b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:11.823245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9106b5b-fcd5-429c-8c13-a637c6c9d476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:11.823862 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9106b5b-fcd5-429c-8c13-a637c6c9d476 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:11.824140 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9106b5b-fcd5-429c-8c13-a637c6c9d476 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:11.824444 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9106b5b-fcd5-429c-8c13-a637c6c9d476 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:11.825230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9106b5b-fcd5-429c-8c13-a637c6c9d476 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:11.825535 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9106b5b-fcd5-429c-8c13-a637c6c9d476 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:11.826956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9106b5b-fcd5-429c-8c13-a637c6c9d476 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:11.844396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 537/1072] 10.222.0.20 () {56 vars in 1133 bytes} [Sat Aug 6 12:19:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:11.849972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adedcbb-9c7b-4f7c-924f-2b34df90c205 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:11.932943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:11.948846 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adedcbb-9c7b-4f7c-924f-2b34df90c205 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:11.981484 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30591779-5e92-4570-afd4-297d063ec48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:11.982111 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30591779-5e92-4570-afd4-297d063ec48a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:11.982387 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30591779-5e92-4570-afd4-297d063ec48a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:11.982657 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30591779-5e92-4570-afd4-297d063ec48a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:11.993699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30591779-5e92-4570-afd4-297d063ec48a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:11.995152 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30591779-5e92-4570-afd4-297d063ec48a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:12.010904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-30591779-5e92-4570-afd4-297d063ec48a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:19:12.010904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:12.010904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:12.010904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:12.010904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:12.010904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:12.010904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:12.010904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:12.010904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:12.010904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:12.010904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:12.010904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:12.010904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:19:12.010904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:19:12.010904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:19:12.012016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:19:12.012016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:12.012016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:12.012016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:12.012016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:12.012016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:12.012016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:12.012016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:12.012016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:12.012016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:12.012016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:12.012016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:12.012016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:12.012016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:12.012016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:12.012016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:12.012016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:12.013093 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:19:12.013093 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:19:12.013093 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:19:12.013093 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:19:12.013328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fcb498-cb94-4cd1-8d58-85e4d77c847f None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:12.013328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fcb498-cb94-4cd1-8d58-85e4d77c847f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:12.013328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fcb498-cb94-4cd1-8d58-85e4d77c847f None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:12.013328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 538/1073] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:19:12.016524 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 536/1074] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:19:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:19:12.017336 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30591779-5e92-4570-afd4-297d063ec48a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:12.017876 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494490c0-9854-4446-98fb-c7a78b00f841 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:12.018156 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494490c0-9854-4446-98fb-c7a78b00f841 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:12.018429 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494490c0-9854-4446-98fb-c7a78b00f841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:12.035669 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-494490c0-9854-4446-98fb-c7a78b00f841 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:12.036871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-494490c0-9854-4446-98fb-c7a78b00f841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:12.087661 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:12.105473 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 539/1075] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 12:19:12.115912 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30591779-5e92-4570-afd4-297d063ec48a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:12.127970 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494490c0-9854-4446-98fb-c7a78b00f841 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:12.150281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3433aa-71dd-4016-bf77-12b2766189d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:12.150892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3433aa-71dd-4016-bf77-12b2766189d4 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:12.151177 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3433aa-71dd-4016-bf77-12b2766189d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:12.151450 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3433aa-71dd-4016-bf77-12b2766189d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:12.152132 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3433aa-71dd-4016-bf77-12b2766189d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:12.152477 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3433aa-71dd-4016-bf77-12b2766189d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:12.153905 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3433aa-71dd-4016-bf77-12b2766189d4 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:12.156106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:12.166740 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 537/1076] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:19:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:12.172222 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3433aa-71dd-4016-bf77-12b2766189d4 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:12.225872 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494490c0-9854-4446-98fb-c7a78b00f841 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:12.260103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44748f67-9e4f-4004-8b05-3ad92a4c907f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:12.260750 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44748f67-9e4f-4004-8b05-3ad92a4c907f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:12.261028 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44748f67-9e4f-4004-8b05-3ad92a4c907f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:12.261298 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44748f67-9e4f-4004-8b05-3ad92a4c907f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:12.268640 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3433aa-71dd-4016-bf77-12b2766189d4 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:12.271077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44748f67-9e4f-4004-8b05-3ad92a4c907f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:19:12.275919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-44748f67-9e4f-4004-8b05-3ad92a4c907f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:19:12.275919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:12.275919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:12.275919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:12.275919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:12.275919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:12.275919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:12.275919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:12.275919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:12.275919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:12.275919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:12.275919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:12.275919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:19:12.275919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:19:12.275919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:19:12.275919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:19:12.276871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:19:12.276871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:19:12.276871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:19:12.276871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:19:12.276871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:12.276871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:12.276871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:12.276871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:12.276871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:12.276871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:12.276871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:12.276871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:12.276871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:12.276871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:12.276871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:12.276871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:12.277752 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:12.277752 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:12.277752 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:12.277752 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:12.277752 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:19:12.277752 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:19:12.277752 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:19:12.277752 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:19:12.277752 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:19:12.277752 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:19:12.277752 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:19:12.277752 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:19:12.277752 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 540/1077] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:19:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:19:12.282942 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44748f67-9e4f-4004-8b05-3ad92a4c907f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:12.302387 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf00d9ee-9c34-4275-a30d-2ddd3ff17c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:12.303012 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00d9ee-9c34-4275-a30d-2ddd3ff17c65 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:12.303293 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00d9ee-9c34-4275-a30d-2ddd3ff17c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:12.304179 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00d9ee-9c34-4275-a30d-2ddd3ff17c65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:12.310743 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf00d9ee-9c34-4275-a30d-2ddd3ff17c65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:12.312183 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf00d9ee-9c34-4275-a30d-2ddd3ff17c65 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:12.323432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-cf00d9ee-9c34-4275-a30d-2ddd3ff17c65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:19:12.323432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:12.323432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:12.323432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:12.323432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:12.323432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:12.323432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:12.323432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:12.323432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:12.323432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:12.323432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:12.323432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:12.323432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:19:12.323432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:19:12.323432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:19:12.324563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:19:12.324563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:12.324563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:12.324563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:12.324563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:12.324563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:12.324563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:12.324563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:12.324563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:12.324563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:12.324563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:12.324563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:12.324563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:12.324563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:12.324563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:12.324563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:12.324563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:12.325577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:19:12.325577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:19:12.325577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:19:12.325577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:19:12.325577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:19:12.325577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:19:12.325577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:19:12.325577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:19:12.325577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 538/1078] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:19:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:19:12.330308 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf00d9ee-9c34-4275-a30d-2ddd3ff17c65 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:12.331271 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:12.386598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44748f67-9e4f-4004-8b05-3ad92a4c907f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:12.386791 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:12.421227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9462095-cb0b-4cd5-b1c1-6e4c91a23a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:12.421842 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9462095-cb0b-4cd5-b1c1-6e4c91a23a4c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:12.422120 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9462095-cb0b-4cd5-b1c1-6e4c91a23a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:12.422393 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9462095-cb0b-4cd5-b1c1-6e4c91a23a4c None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:12.423103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9462095-cb0b-4cd5-b1c1-6e4c91a23a4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:12.423404 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9462095-cb0b-4cd5-b1c1-6e4c91a23a4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:12.424800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9462095-cb0b-4cd5-b1c1-6e4c91a23a4c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:12.430824 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf00d9ee-9c34-4275-a30d-2ddd3ff17c65 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:12.433106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 541/1079] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:19:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:12.438924 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9462095-cb0b-4cd5-b1c1-6e4c91a23a4c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:12.464535 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db627f-ba9d-482c-b05a-b5c957e7b40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:12.465141 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db627f-ba9d-482c-b05a-b5c957e7b40d None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:12.465413 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db627f-ba9d-482c-b05a-b5c957e7b40d None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:12.465682 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db627f-ba9d-482c-b05a-b5c957e7b40d None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:12.466421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db627f-ba9d-482c-b05a-b5c957e7b40d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:12.466723 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db627f-ba9d-482c-b05a-b5c957e7b40d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:12.468139 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db627f-ba9d-482c-b05a-b5c957e7b40d None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:12.480114 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 539/1080] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:19:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:12.485946 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db627f-ba9d-482c-b05a-b5c957e7b40d None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:12.536650 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9462095-cb0b-4cd5-b1c1-6e4c91a23a4c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:12.546239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:12.570436 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efccad-87f6-4b7e-b957-b435afc539de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:12.571062 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efccad-87f6-4b7e-b957-b435afc539de None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:12.571341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efccad-87f6-4b7e-b957-b435afc539de None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:12.571614 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efccad-87f6-4b7e-b957-b435afc539de None admin] PATH_INFO: `/v3/users/0459782d36d842f58a2f95f6167a229f` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:12.572803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0efccad-87f6-4b7e-b957-b435afc539de None admin] RBAC: Authorizing `identity:get_user(user_id=0459782d36d842f58a2f95f6167a229f)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:12.574215 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0efccad-87f6-4b7e-b957-b435afc539de None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:12.575818 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 542/1081] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:19:12 2022] GET /identity/v3/users/0459782d36d842f58a2f95f6167a229f => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:12.581372 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efccad-87f6-4b7e-b957-b435afc539de None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:12.583554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db627f-ba9d-482c-b05a-b5c957e7b40d None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:12.600165 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:12.619501 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d5bf4-649b-414b-b16b-47a41e80b554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:12.620207 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d5bf4-649b-414b-b16b-47a41e80b554 None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:12.620523 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d5bf4-649b-414b-b16b-47a41e80b554 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:12.620807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d5bf4-649b-414b-b16b-47a41e80b554 None admin] PATH_INFO: `/v3/projects/13fba33c46cf40bbb15a08c690b4c98c/users/524b728bf63c4ca88e0f25e531cf5056/roles/ed864363c041447ab55d1795a8c7092b` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:12.623046 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d5bf4-649b-414b-b16b-47a41e80b554 None admin] RBAC: Authorizing `identity:create_grant(project_id=13fba33c46cf40bbb15a08c690b4c98c, user_id=524b728bf63c4ca88e0f25e531cf5056, role_id=ed864363c041447ab55d1795a8c7092b)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:12.624502 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d5bf4-649b-414b-b16b-47a41e80b554 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:12.637692 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 540/1082] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:19:12 2022] PUT /identity/v3/projects/13fba33c46cf40bbb15a08c690b4c98c/users/524b728bf63c4ca88e0f25e531cf5056/roles/ed864363c041447ab55d1795a8c7092b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:19:12.683252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efccad-87f6-4b7e-b957-b435afc539de None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:12.719898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf916d4-373c-49e2-9d78-b512cae4a0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:12.720584 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf916d4-373c-49e2-9d78-b512cae4a0d4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:12.720862 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf916d4-373c-49e2-9d78-b512cae4a0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:12.721139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf916d4-373c-49e2-9d78-b512cae4a0d4 None admin] PATH_INFO: `/v3/projects/db2b557448f04e4aabfb2859cd1be4a1` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:12.722314 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf916d4-373c-49e2-9d78-b512cae4a0d4 None admin] RBAC: Authorizing `identity:get_project(project_id=db2b557448f04e4aabfb2859cd1be4a1)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:12.723758 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf916d4-373c-49e2-9d78-b512cae4a0d4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:12.725569 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 543/1083] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:19:12 2022] GET /identity/v3/projects/db2b557448f04e4aabfb2859cd1be4a1 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:12.731713 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d5bf4-649b-414b-b16b-47a41e80b554 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:12.833556 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d5bf4-649b-414b-b16b-47a41e80b554 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:12.867278 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d9a66-db9c-412d-a324-48966493c39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:12.867933 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d9a66-db9c-412d-a324-48966493c39e None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:12.868217 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d9a66-db9c-412d-a324-48966493c39e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:12.868521 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d9a66-db9c-412d-a324-48966493c39e None admin] PATH_INFO: `/v3/projects/db2b557448f04e4aabfb2859cd1be4a1/users/0459782d36d842f58a2f95f6167a229f/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:12.870455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d9a66-db9c-412d-a324-48966493c39e None admin] RBAC: Authorizing `identity:create_grant(project_id=db2b557448f04e4aabfb2859cd1be4a1, user_id=0459782d36d842f58a2f95f6167a229f, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:12.871883 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d9a66-db9c-412d-a324-48966493c39e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:12.884576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 541/1084] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:19:12 2022] PUT /identity/v3/projects/db2b557448f04e4aabfb2859cd1be4a1/users/0459782d36d842f58a2f95f6167a229f/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:19:14.620116 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc215d-e83c-46ef-b6b2-61929cdc00f8 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:14.620772 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc215d-e83c-46ef-b6b2-61929cdc00f8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:14.620772 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc215d-e83c-46ef-b6b2-61929cdc00f8 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:14.622032 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 544/1085] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:19:14.627067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb52ce-ba30-4e87-b40a-0057e218f433 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:14.627456 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb52ce-ba30-4e87-b40a-0057e218f433 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:14.627746 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb52ce-ba30-4e87-b40a-0057e218f433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:14.645979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-d8eb52ce-ba30-4e87-b40a-0057e218f433 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:14.647668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d8eb52ce-ba30-4e87-b40a-0057e218f433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:14.675640 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:14.713683 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 542/1086] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:19:14.737019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc215d-e83c-46ef-b6b2-61929cdc00f8 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:14.783889 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:14.838143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc215d-e83c-46ef-b6b2-61929cdc00f8 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:14.860547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddbea9-fd6d-4fec-86ab-6016e1e0b792 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:14.860844 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddbea9-fd6d-4fec-86ab-6016e1e0b792 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:14.861117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddbea9-fd6d-4fec-86ab-6016e1e0b792 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:14.862478 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 543/1087] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:19:14.867980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb6086-889c-462a-b34d-09a27d31bf2a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:14.874311 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb6086-889c-462a-b34d-09a27d31bf2a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:14.874453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb6086-889c-462a-b34d-09a27d31bf2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:14.876910 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565debcd-3603-4554-97d5-4ef191c254a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:14.877565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565debcd-3603-4554-97d5-4ef191c254a8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:14.877966 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565debcd-3603-4554-97d5-4ef191c254a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:14.878244 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565debcd-3603-4554-97d5-4ef191c254a8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:14.888439 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565debcd-3603-4554-97d5-4ef191c254a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:19:14.891629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-d0bb6086-889c-462a-b34d-09a27d31bf2a None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:14.892815 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d0bb6086-889c-462a-b34d-09a27d31bf2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:14.893330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-565debcd-3603-4554-97d5-4ef191c254a8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 12:19:14.893330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:14.893330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:14.893330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:14.893330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:14.893330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:14.893330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:14.893330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:14.893330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:14.893330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:14.893330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:14.893330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:14.893330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:19:14.893330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:19:14.893330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:19:14.893330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:19:14.894245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:19:14.894245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:19:14.894245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:19:14.894245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:19:14.894245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:14.894245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:14.894245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:14.894245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:14.894245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:14.894245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:14.894245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:14.894245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:14.894245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:14.894245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:14.894245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:14.894245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:14.895103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:14.895103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:14.895103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:14.895103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:14.895103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:19:14.895103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:19:14.895103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:19:14.895103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:19:14.895103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:19:14.895103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:19:14.895103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 06 12:19:14.895103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:19:14.895103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 545/1088] 10.222.0.20 () {56 vars in 1120 bytes} [Sat Aug 6 12:19:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:19:14.900901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565debcd-3603-4554-97d5-4ef191c254a8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:14.958948 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 544/1089] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 12:19:14.983361 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb6086-889c-462a-b34d-09a27d31bf2a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:15.001006 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565debcd-3603-4554-97d5-4ef191c254a8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:15.003304 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:15.035835 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216bffdb-ca91-4583-97d9-0c47720a8e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:15.036487 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216bffdb-ca91-4583-97d9-0c47720a8e44 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:15.036766 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216bffdb-ca91-4583-97d9-0c47720a8e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:15.037037 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216bffdb-ca91-4583-97d9-0c47720a8e44 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:15.037750 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216bffdb-ca91-4583-97d9-0c47720a8e44 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:15.038048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216bffdb-ca91-4583-97d9-0c47720a8e44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:15.039399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216bffdb-ca91-4583-97d9-0c47720a8e44 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:15.047104 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 546/1090] 10.222.0.20 () {56 vars in 1115 bytes} [Sat Aug 6 12:19:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:15.052774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216bffdb-ca91-4583-97d9-0c47720a8e44 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:15.076878 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:15.083000 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb6086-889c-462a-b34d-09a27d31bf2a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:15.116802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeee702-0b3c-44e7-9b11-d886fc7519fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:15.117436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeee702-0b3c-44e7-9b11-d886fc7519fe None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:15.117724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeee702-0b3c-44e7-9b11-d886fc7519fe None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:15.118017 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeee702-0b3c-44e7-9b11-d886fc7519fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:15.128139 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeee702-0b3c-44e7-9b11-d886fc7519fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:19:15.133591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-dfeee702-0b3c-44e7-9b11-d886fc7519fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:19:15.133591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:15.133591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:15.133591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:15.133591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:15.133591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:15.133591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:15.133591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:15.133591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:15.133591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:15.133591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:15.133591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:15.133591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:19:15.133591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:19:15.133591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:19:15.133591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:19:15.134710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:19:15.134710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:19:15.134710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:19:15.134710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:19:15.134710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:15.134710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:15.134710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:15.134710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:15.134710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:15.134710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:15.134710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:15.134710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:15.134710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:15.134710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:15.134710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:15.134710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:15.135916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:15.135916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:15.135916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:15.135916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:15.135916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:19:15.135916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:19:15.135916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:19:15.135916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:19:15.135916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:19:15.135916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:19:15.135916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:19:15.135916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:19:15.135916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 545/1091] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:19:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:19:15.140723 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeee702-0b3c-44e7-9b11-d886fc7519fe None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:15.153828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216bffdb-ca91-4583-97d9-0c47720a8e44 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:15.187014 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f1a4b7-184d-47cb-a5eb-4951df8cdffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:15.187652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1a4b7-184d-47cb-a5eb-4951df8cdffe None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:15.187937 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1a4b7-184d-47cb-a5eb-4951df8cdffe None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:15.188207 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1a4b7-184d-47cb-a5eb-4951df8cdffe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:15.199480 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f1a4b7-184d-47cb-a5eb-4951df8cdffe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:15.200961 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f1a4b7-184d-47cb-a5eb-4951df8cdffe None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:15.215613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-c2f1a4b7-184d-47cb-a5eb-4951df8cdffe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:19:15.215613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:15.215613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:15.215613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:15.215613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:15.215613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:15.215613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:15.215613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:15.215613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:15.215613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:15.215613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:15.215613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:15.215613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:19:15.215613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:19:15.215613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:19:15.216646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:19:15.216646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:15.216646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:15.216646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:15.216646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:15.216646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:15.216646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:15.216646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:15.216646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:15.216646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:15.216646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:15.216646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:15.216646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:15.216646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:15.216646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:15.216646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:15.216646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:15.217560 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:19:15.217560 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:19:15.217560 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:19:15.217560 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:19:15.217560 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 547/1092] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:19:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:19:15.222539 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f1a4b7-184d-47cb-a5eb-4951df8cdffe None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:15.237198 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeee702-0b3c-44e7-9b11-d886fc7519fe None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:15.241237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:15.271645 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885ec17-16a6-4e9b-a6c0-f8b94dec6d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:15.272848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885ec17-16a6-4e9b-a6c0-f8b94dec6d17 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:15.273128 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885ec17-16a6-4e9b-a6c0-f8b94dec6d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:15.273400 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885ec17-16a6-4e9b-a6c0-f8b94dec6d17 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:15.274098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885ec17-16a6-4e9b-a6c0-f8b94dec6d17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:15.274404 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885ec17-16a6-4e9b-a6c0-f8b94dec6d17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:15.275759 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885ec17-16a6-4e9b-a6c0-f8b94dec6d17 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:15.283538 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 546/1093] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:19:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:15.289269 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885ec17-16a6-4e9b-a6c0-f8b94dec6d17 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:15.307234 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:15.322782 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f1a4b7-184d-47cb-a5eb-4951df8cdffe None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:15.355808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfd90f-ae65-4273-93a5-dba3eca608dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:15.356446 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfd90f-ae65-4273-93a5-dba3eca608dd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:15.356726 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfd90f-ae65-4273-93a5-dba3eca608dd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:15.356996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfd90f-ae65-4273-93a5-dba3eca608dd None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:15.357691 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfd90f-ae65-4273-93a5-dba3eca608dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:15.357986 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfd90f-ae65-4273-93a5-dba3eca608dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:15.359332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfd90f-ae65-4273-93a5-dba3eca608dd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:15.371200 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 548/1094] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:15.376744 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfd90f-ae65-4273-93a5-dba3eca608dd None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:15.387977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885ec17-16a6-4e9b-a6c0-f8b94dec6d17 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:15.421751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3d2ec9-8908-4983-a158-dbc7ea222d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:15.422568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d2ec9-8908-4983-a158-dbc7ea222d0e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:15.422846 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d2ec9-8908-4983-a158-dbc7ea222d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:15.423120 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d2ec9-8908-4983-a158-dbc7ea222d0e None admin] PATH_INFO: `/v3/users/0459782d36d842f58a2f95f6167a229f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:15.424279 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3d2ec9-8908-4983-a158-dbc7ea222d0e None admin] RBAC: Authorizing `identity:get_user(user_id=0459782d36d842f58a2f95f6167a229f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:15.425764 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3d2ec9-8908-4983-a158-dbc7ea222d0e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:15.427348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 547/1095] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:19:15 2022] GET /identity/v3/users/0459782d36d842f58a2f95f6167a229f => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:15.432994 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3d2ec9-8908-4983-a158-dbc7ea222d0e None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:15.453250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:15.476571 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfd90f-ae65-4273-93a5-dba3eca608dd None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:15.509869 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c198c2-2f3b-4246-877d-d9258d7bee1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:15.510504 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c198c2-2f3b-4246-877d-d9258d7bee1c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:15.510782 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c198c2-2f3b-4246-877d-d9258d7bee1c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:15.511055 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c198c2-2f3b-4246-877d-d9258d7bee1c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:15.518015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c198c2-2f3b-4246-877d-d9258d7bee1c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:15.518792 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:15.519604 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c198c2-2f3b-4246-877d-d9258d7bee1c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:15.531886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-47c198c2-2f3b-4246-877d-d9258d7bee1c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 06 12:19:15.531886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:15.531886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:15.531886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:15.531886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:15.531886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:15.531886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:15.531886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:15.531886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:15.531886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:15.531886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:15.531886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:15.531886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:19:15.531886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:19:15.531886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:19:15.531886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:19:15.532877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:15.532877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:15.532877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:19:15.532877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:19:15.532877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:19:15.532877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:19:15.532877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:15.532877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:15.532877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:15.532877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:15.532877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:15.532877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:15.532877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:15.532877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:15.532877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:15.532877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:15.532877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:15.532877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:15.533990 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:15.533990 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:15.533990 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:19:15.533990 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:19:15.533990 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:19:15.533990 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:19:15.533990 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:19:15.533990 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:19:15.533990 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 06 12:19:15.533990 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:19:15.533990 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 549/1096] 10.222.0.20 () {56 vars in 1093 bytes} [Sat Aug 6 12:19:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:19:15.539238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c198c2-2f3b-4246-877d-d9258d7bee1c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:15.539352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3d2ec9-8908-4983-a158-dbc7ea222d0e None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:15.572847 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55ac78-88dc-4bd8-a7bd-3f2f3a7b08f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:15.573450 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55ac78-88dc-4bd8-a7bd-3f2f3a7b08f5 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:15.573724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55ac78-88dc-4bd8-a7bd-3f2f3a7b08f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:15.573995 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55ac78-88dc-4bd8-a7bd-3f2f3a7b08f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:15.636936 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c198c2-2f3b-4246-877d-d9258d7bee1c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:15.639704 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55ac78-88dc-4bd8-a7bd-3f2f3a7b08f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:15.641214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55ac78-88dc-4bd8-a7bd-3f2f3a7b08f5 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:15.673982 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6de65-42bd-4075-9360-a10846f36fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:15.674587 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6de65-42bd-4075-9360-a10846f36fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:15.674865 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6de65-42bd-4075-9360-a10846f36fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:15.675137 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6de65-42bd-4075-9360-a10846f36fa1 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:15.675912 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d6de65-42bd-4075-9360-a10846f36fa1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:15.676216 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d6de65-42bd-4075-9360-a10846f36fa1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:15.677638 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d6de65-42bd-4075-9360-a10846f36fa1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:15.685921 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:15.697125 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 550/1097] 10.222.0.20 () {56 vars in 1133 bytes} [Sat Aug 6 12:19:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:15.702734 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6de65-42bd-4075-9360-a10846f36fa1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:15.723527 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:15.739208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 548/1098] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:19:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 06 12:19:15.745830 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55ac78-88dc-4bd8-a7bd-3f2f3a7b08f5 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:15.800679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6de65-42bd-4075-9360-a10846f36fa1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:15.833486 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85e762-98a8-4e6a-b38d-79352dded4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:15.834091 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85e762-98a8-4e6a-b38d-79352dded4ba None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:15.834365 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85e762-98a8-4e6a-b38d-79352dded4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:15.835172 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85e762-98a8-4e6a-b38d-79352dded4ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:15.842871 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55ac78-88dc-4bd8-a7bd-3f2f3a7b08f5 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:15.875367 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65efb04-c063-43a2-9114-0a639527406c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:15.876067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65efb04-c063-43a2-9114-0a639527406c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:15.876381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65efb04-c063-43a2-9114-0a639527406c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:15.876657 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65efb04-c063-43a2-9114-0a639527406c None admin] PATH_INFO: `/v3/projects/db2b557448f04e4aabfb2859cd1be4a1` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:15.877799 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65efb04-c063-43a2-9114-0a639527406c None admin] RBAC: Authorizing `identity:get_project(project_id=db2b557448f04e4aabfb2859cd1be4a1)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:15.879229 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65efb04-c063-43a2-9114-0a639527406c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:15.880897 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 549/1099] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:19:15 2022] GET /identity/v3/projects/db2b557448f04e4aabfb2859cd1be4a1 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:15.887422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65efb04-c063-43a2-9114-0a639527406c None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:15.895853 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:15.903257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85e762-98a8-4e6a-b38d-79352dded4ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:15.904739 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85e762-98a8-4e6a-b38d-79352dded4ba None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:15.941699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:15.984651 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65efb04-c063-43a2-9114-0a639527406c None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:16.000657 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 551/1100] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:19:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 12:19:16.006033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85e762-98a8-4e6a-b38d-79352dded4ba None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:16.025889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1ed7a-754b-4a94-820e-c08ee5b787af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:16.026505 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1ed7a-754b-4a94-820e-c08ee5b787af None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:16.026773 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1ed7a-754b-4a94-820e-c08ee5b787af None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:16.027040 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1ed7a-754b-4a94-820e-c08ee5b787af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:16.027814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a1ed7a-754b-4a94-820e-c08ee5b787af None admin] RBAC: Adding query filter params (role.id=144c8425688b409384e1e8ed8279d88e, scope.project.id=db2b557448f04e4aabfb2859cd1be4a1, user.id=0459782d36d842f58a2f95f6167a229f) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:16.028118 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a1ed7a-754b-4a94-820e-c08ee5b787af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:16.029578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a1ed7a-754b-4a94-820e-c08ee5b787af None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:16.035156 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 550/1101] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:19:15 2022] GET /identity/v3/role_assignments?user.id=0459782d36d842f58a2f95f6167a229f&scope.project.id=db2b557448f04e4aabfb2859cd1be4a1&role.id=144c8425688b409384e1e8ed8279d88e => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:16.105726 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:16.112207 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85e762-98a8-4e6a-b38d-79352dded4ba None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:16.147587 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e8539-4ada-4eae-8388-959e3b031f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:16.148152 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e8539-4ada-4eae-8388-959e3b031f27 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:16.148460 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e8539-4ada-4eae-8388-959e3b031f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:16.148732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e8539-4ada-4eae-8388-959e3b031f27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:16.159581 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e8539-4ada-4eae-8388-959e3b031f27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:16.161062 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e8539-4ada-4eae-8388-959e3b031f27 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:16.176951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-216e8539-4ada-4eae-8388-959e3b031f27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:19:16.176951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:16.176951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:16.176951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:16.176951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:16.176951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:16.176951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:16.176951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:16.176951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:16.176951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:16.176951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:16.176951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:16.176951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:19:16.176951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:19:16.176951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:19:16.177906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:19:16.177906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:16.177906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:16.177906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:16.177906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:16.177906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:16.177906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:16.177906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:16.177906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:16.177906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:16.177906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:16.177906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:16.177906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:16.177906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:16.177906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:16.177906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:16.177906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:16.178978 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:19:16.178978 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:19:16.178978 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:19:16.178978 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:19:16.178978 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 552/1102] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:19:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:19:16.183891 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1ed7a-754b-4a94-820e-c08ee5b787af None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:16.284434 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1ed7a-754b-4a94-820e-c08ee5b787af None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:16.301254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:16.317976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0910889-4476-43df-8a13-4419cdb722ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:16.318584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0910889-4476-43df-8a13-4419cdb722ec None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:16.318916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0910889-4476-43df-8a13-4419cdb722ec None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:16.319198 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0910889-4476-43df-8a13-4419cdb722ec None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:16.319880 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0910889-4476-43df-8a13-4419cdb722ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:16.320183 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0910889-4476-43df-8a13-4419cdb722ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:16.321589 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0910889-4476-43df-8a13-4419cdb722ec None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:16.332924 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 551/1103] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:16.338899 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e8539-4ada-4eae-8388-959e3b031f27 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:16.436493 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e8539-4ada-4eae-8388-959e3b031f27 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:16.468699 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef11ba-1c7f-4d40-90ce-d958cfa52c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:16.469331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef11ba-1c7f-4d40-90ce-d958cfa52c9a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:16.469609 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef11ba-1c7f-4d40-90ce-d958cfa52c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:16.469879 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef11ba-1c7f-4d40-90ce-d958cfa52c9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:16.476960 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef11ba-1c7f-4d40-90ce-d958cfa52c9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:16.478385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef11ba-1c7f-4d40-90ce-d958cfa52c9a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:16.489466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-a3ef11ba-1c7f-4d40-90ce-d958cfa52c9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:19:16.489466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:16.489466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:16.489466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:16.489466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:16.489466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:16.489466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:16.489466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:16.489466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:16.489466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:16.489466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:16.489466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:16.489466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:19:16.489466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:19:16.489466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:19:16.490491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:19:16.490491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:16.490491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:16.490491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:16.490491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:16.490491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:16.490491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:16.490491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:16.490491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:16.490491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:16.490491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:16.490491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:16.490491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:16.490491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:16.490491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:16.490491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:16.490491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:16.491653 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:19:16.491653 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:19:16.491653 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:19:16.491653 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:19:16.491653 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:19:16.491653 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:19:16.491653 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:19:16.491653 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:19:16.491653 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 553/1104] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:19:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:19:16.496818 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0910889-4476-43df-8a13-4419cdb722ec None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:16.595438 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0910889-4476-43df-8a13-4419cdb722ec None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:16.628067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924c5c2-b2f7-49bf-9b08-2ed7fe21e8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:16.628705 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924c5c2-b2f7-49bf-9b08-2ed7fe21e8ed None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:16.628983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924c5c2-b2f7-49bf-9b08-2ed7fe21e8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:16.629253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924c5c2-b2f7-49bf-9b08-2ed7fe21e8ed None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:16.629993 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924c5c2-b2f7-49bf-9b08-2ed7fe21e8ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:16.630295 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924c5c2-b2f7-49bf-9b08-2ed7fe21e8ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:16.632246 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924c5c2-b2f7-49bf-9b08-2ed7fe21e8ed None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:16.643527 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 552/1105] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:19:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:16.649992 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef11ba-1c7f-4d40-90ce-d958cfa52c9a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:16.661329 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:16.747881 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef11ba-1c7f-4d40-90ce-d958cfa52c9a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:16.782039 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd3d91-07d6-470e-8b89-4f22ba2663bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:16.782657 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd3d91-07d6-470e-8b89-4f22ba2663bb None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:16.782937 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd3d91-07d6-470e-8b89-4f22ba2663bb None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:16.783210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd3d91-07d6-470e-8b89-4f22ba2663bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:16.784006 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd3d91-07d6-470e-8b89-4f22ba2663bb None admin] RBAC: Adding query filter params (role.id=ed864363c041447ab55d1795a8c7092b, scope.project.id=13fba33c46cf40bbb15a08c690b4c98c, user.id=524b728bf63c4ca88e0f25e531cf5056) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:16.784332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd3d91-07d6-470e-8b89-4f22ba2663bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:16.786901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd3d91-07d6-470e-8b89-4f22ba2663bb None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:16.794677 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 554/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:19:16 2022] GET /identity/v3/role_assignments?user.id=524b728bf63c4ca88e0f25e531cf5056&scope.project.id=13fba33c46cf40bbb15a08c690b4c98c&role.id=ed864363c041447ab55d1795a8c7092b => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:18.039989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352594f-751a-46ee-9a4e-64228bbe106d None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:18.040547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352594f-751a-46ee-9a4e-64228bbe106d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:18.040547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352594f-751a-46ee-9a4e-64228bbe106d None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:18.041885 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 553/1107] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:19:18.046940 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ca100-ecda-4528-b413-e5e9a4144a97 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:18.047357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ca100-ecda-4528-b413-e5e9a4144a97 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:18.047489 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ca100-ecda-4528-b413-e5e9a4144a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:18.065755 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a61ca100-ecda-4528-b413-e5e9a4144a97 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:18.066923 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a61ca100-ecda-4528-b413-e5e9a4144a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:18.129969 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 555/1108] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:19:18.153888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352594f-751a-46ee-9a4e-64228bbe106d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:18.193099 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:18.252427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352594f-751a-46ee-9a4e-64228bbe106d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:18.285244 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a25b9-4cd9-42ec-bd0a-c7fea6aa6e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:18.285859 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a25b9-4cd9-42ec-bd0a-c7fea6aa6e85 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:18.286132 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a25b9-4cd9-42ec-bd0a-c7fea6aa6e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:18.286399 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a25b9-4cd9-42ec-bd0a-c7fea6aa6e85 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:18.287542 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a25b9-4cd9-42ec-bd0a-c7fea6aa6e85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:18.288999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a25b9-4cd9-42ec-bd0a-c7fea6aa6e85 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:18.290553 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 554/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:19:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:18.296289 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ca100-ecda-4528-b413-e5e9a4144a97 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:18.298482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:18.392628 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ca100-ecda-4528-b413-e5e9a4144a97 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:18.424048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f2e5e-f2d5-446f-a601-72564db37bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:18.424778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f2e5e-f2d5-446f-a601-72564db37bb9 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:18.425053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f2e5e-f2d5-446f-a601-72564db37bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:18.425319 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f2e5e-f2d5-446f-a601-72564db37bb9 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:18.425984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f2e5e-f2d5-446f-a601-72564db37bb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:18.427384 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f2e5e-f2d5-446f-a601-72564db37bb9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:18.447194 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 556/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Aug 6 12:19:18 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:19:20.458992 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a8af1-0e7e-40ae-a443-10ab2ebe44c6 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:20.459622 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a8af1-0e7e-40ae-a443-10ab2ebe44c6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:20.459622 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a8af1-0e7e-40ae-a443-10ab2ebe44c6 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:20.460899 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 555/1111] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:19:20.466090 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2ebb6-985b-48cd-97d3-bb69b596eb42 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:20.466519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2ebb6-985b-48cd-97d3-bb69b596eb42 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:20.466622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2ebb6-985b-48cd-97d3-bb69b596eb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:20.484819 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-aca2ebb6-985b-48cd-97d3-bb69b596eb42 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:20.485988 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-aca2ebb6-985b-48cd-97d3-bb69b596eb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:20.537839 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:20.551534 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 557/1112] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:19:20.574791 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a8af1-0e7e-40ae-a443-10ab2ebe44c6 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:20.673821 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a8af1-0e7e-40ae-a443-10ab2ebe44c6 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:20.681312 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:20.707066 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da338d5-0a95-4efb-8198-2284c5c560ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:20.707684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da338d5-0a95-4efb-8198-2284c5c560ae None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:20.707959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da338d5-0a95-4efb-8198-2284c5c560ae None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:20.708231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da338d5-0a95-4efb-8198-2284c5c560ae None admin] PATH_INFO: `/v3/roles/47cde553d7ee465bbe685a05c00b94e2` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:20.709739 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da338d5-0a95-4efb-8198-2284c5c560ae None admin] RBAC: Authorizing `identity:get_role(role_id=47cde553d7ee465bbe685a05c00b94e2)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:20.711052 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da338d5-0a95-4efb-8198-2284c5c560ae None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:20.712372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 556/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:19:20 2022] GET /identity/v3/roles/47cde553d7ee465bbe685a05c00b94e2 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:20.717989 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca2ebb6-985b-48cd-97d3-bb69b596eb42 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:20.812139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca2ebb6-985b-48cd-97d3-bb69b596eb42 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:20.844254 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02a662-bfea-432b-8fbe-6dbd57c68f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:20.844964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02a662-bfea-432b-8fbe-6dbd57c68f76 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:20.845240 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02a662-bfea-432b-8fbe-6dbd57c68f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:20.845508 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02a662-bfea-432b-8fbe-6dbd57c68f76 None admin] PATH_INFO: `/v3/users/c1a5cce2bb1c42159d0adad5cd0c48ba` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:20.857930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02a662-bfea-432b-8fbe-6dbd57c68f76 None admin] RBAC: Authorizing `identity:get_user(user_id=c1a5cce2bb1c42159d0adad5cd0c48ba)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:20.859337 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02a662-bfea-432b-8fbe-6dbd57c68f76 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:20.860950 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 558/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:19:20 2022] GET /identity/v3/users/c1a5cce2bb1c42159d0adad5cd0c48ba => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:20.867269 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da338d5-0a95-4efb-8198-2284c5c560ae None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:20.961016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da338d5-0a95-4efb-8198-2284c5c560ae None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:20.993226 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9310b5d-2735-4980-9263-88039f982ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:20.993836 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9310b5d-2735-4980-9263-88039f982ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:20.994111 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9310b5d-2735-4980-9263-88039f982ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:20.994381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9310b5d-2735-4980-9263-88039f982ed2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:21.038660 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:21.063642 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9310b5d-2735-4980-9263-88039f982ed2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:21.065135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9310b5d-2735-4980-9263-88039f982ed2 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:21.157660 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 557/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:19:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 12:19:21.164108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02a662-bfea-432b-8fbe-6dbd57c68f76 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:21.209247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:21.260808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02a662-bfea-432b-8fbe-6dbd57c68f76 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:21.293791 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1d126-34fd-4119-b5ff-efcccc17f9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:21.294413 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1d126-34fd-4119-b5ff-efcccc17f9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:21.294690 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1d126-34fd-4119-b5ff-efcccc17f9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:21.294961 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1d126-34fd-4119-b5ff-efcccc17f9f0 None admin] PATH_INFO: `/v3/projects/db2b557448f04e4aabfb2859cd1be4a1` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:21.296099 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1d126-34fd-4119-b5ff-efcccc17f9f0 None admin] RBAC: Authorizing `identity:get_project(project_id=db2b557448f04e4aabfb2859cd1be4a1)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:21.297557 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1d126-34fd-4119-b5ff-efcccc17f9f0 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:21.299198 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 559/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:19:21 2022] GET /identity/v3/projects/db2b557448f04e4aabfb2859cd1be4a1 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:21.305763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9310b5d-2735-4980-9263-88039f982ed2 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:21.380930 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:21.405499 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9310b5d-2735-4980-9263-88039f982ed2 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:21.437523 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13570b18-30ea-4ee4-a084-209d454e35cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:21.438120 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13570b18-30ea-4ee4-a084-209d454e35cc None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:21.438395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13570b18-30ea-4ee4-a084-209d454e35cc None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:21.438666 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13570b18-30ea-4ee4-a084-209d454e35cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:21.439433 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13570b18-30ea-4ee4-a084-209d454e35cc None admin] RBAC: Adding query filter params (role.id=47cde553d7ee465bbe685a05c00b94e2, scope.project.id=db2b557448f04e4aabfb2859cd1be4a1, user.id=c1a5cce2bb1c42159d0adad5cd0c48ba) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:21.439738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13570b18-30ea-4ee4-a084-209d454e35cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:21.441187 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13570b18-30ea-4ee4-a084-209d454e35cc None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:21.446316 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 558/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:19:21 2022] GET /identity/v3/role_assignments?user.id=c1a5cce2bb1c42159d0adad5cd0c48ba&scope.project.id=db2b557448f04e4aabfb2859cd1be4a1&role.id=47cde553d7ee465bbe685a05c00b94e2 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:23.389840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a9a3b-ab00-4bd1-a113-f7df19812598 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:23.390392 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a9a3b-ab00-4bd1-a113-f7df19812598 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:23.390392 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a9a3b-ab00-4bd1-a113-f7df19812598 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:23.391724 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 560/1118] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:19:23.397519 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08633a01-87f4-49ba-8b67-ca926d818dd7 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:23.397924 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08633a01-87f4-49ba-8b67-ca926d818dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:23.398059 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08633a01-87f4-49ba-8b67-ca926d818dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:23.415480 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-08633a01-87f4-49ba-8b67-ca926d818dd7 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:23.416664 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-08633a01-87f4-49ba-8b67-ca926d818dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:23.479543 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 559/1119] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:19:23.501620 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a9a3b-ab00-4bd1-a113-f7df19812598 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:23.600186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a9a3b-ab00-4bd1-a113-f7df19812598 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:23.613900 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:23.635300 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61918cd-0714-4861-98c7-eb3a431619cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:23.635931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61918cd-0714-4861-98c7-eb3a431619cc None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:23.636210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61918cd-0714-4861-98c7-eb3a431619cc None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:23.636513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61918cd-0714-4861-98c7-eb3a431619cc None admin] PATH_INFO: `/v3/roles/47cde553d7ee465bbe685a05c00b94e2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:23.637936 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61918cd-0714-4861-98c7-eb3a431619cc None admin] RBAC: Authorizing `identity:get_role(role_id=47cde553d7ee465bbe685a05c00b94e2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:23.639290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61918cd-0714-4861-98c7-eb3a431619cc None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:23.640717 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 561/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:19:23 2022] GET /identity/v3/roles/47cde553d7ee465bbe685a05c00b94e2 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:23.646330 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08633a01-87f4-49ba-8b67-ca926d818dd7 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:23.708598 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:23.739424 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08633a01-87f4-49ba-8b67-ca926d818dd7 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:23.773177 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c1bb7-22cb-49b2-be8b-bcf92f6333b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:23.773796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c1bb7-22cb-49b2-be8b-bcf92f6333b0 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:23.774069 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c1bb7-22cb-49b2-be8b-bcf92f6333b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:23.774337 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c1bb7-22cb-49b2-be8b-bcf92f6333b0 None admin] PATH_INFO: `/v3/users/c1a5cce2bb1c42159d0adad5cd0c48ba` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:23.775461 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c1bb7-22cb-49b2-be8b-bcf92f6333b0 None admin] RBAC: Authorizing `identity:get_user(user_id=c1a5cce2bb1c42159d0adad5cd0c48ba)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:23.776912 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c1bb7-22cb-49b2-be8b-bcf92f6333b0 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:23.778464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 560/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:19:23 2022] GET /identity/v3/users/c1a5cce2bb1c42159d0adad5cd0c48ba => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:23.784018 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61918cd-0714-4861-98c7-eb3a431619cc None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:23.885940 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61918cd-0714-4861-98c7-eb3a431619cc None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:23.917925 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43ffea-d20c-4127-b22a-ec7609e6e62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:23.918479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43ffea-d20c-4127-b22a-ec7609e6e62a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:23.918756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43ffea-d20c-4127-b22a-ec7609e6e62a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:23.919025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43ffea-d20c-4127-b22a-ec7609e6e62a None admin] PATH_INFO: `/v3/projects/db2b557448f04e4aabfb2859cd1be4a1` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:23.920164 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43ffea-d20c-4127-b22a-ec7609e6e62a None admin] RBAC: Authorizing `identity:get_project(project_id=db2b557448f04e4aabfb2859cd1be4a1)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:23.921597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43ffea-d20c-4127-b22a-ec7609e6e62a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:23.923235 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 562/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:19:23 2022] GET /identity/v3/projects/db2b557448f04e4aabfb2859cd1be4a1 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:23.929618 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c1bb7-22cb-49b2-be8b-bcf92f6333b0 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:24.024641 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c1bb7-22cb-49b2-be8b-bcf92f6333b0 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:24.057572 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d92bd-8792-4f94-80e3-a52b374bbc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:24.058203 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d92bd-8792-4f94-80e3-a52b374bbc4e None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:24.058482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d92bd-8792-4f94-80e3-a52b374bbc4e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:24.058750 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d92bd-8792-4f94-80e3-a52b374bbc4e None admin] PATH_INFO: `/v3/projects/db2b557448f04e4aabfb2859cd1be4a1/users/c1a5cce2bb1c42159d0adad5cd0c48ba/roles/47cde553d7ee465bbe685a05c00b94e2` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:24.060888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2d92bd-8792-4f94-80e3-a52b374bbc4e None admin] RBAC: Authorizing `identity:create_grant(project_id=db2b557448f04e4aabfb2859cd1be4a1, user_id=c1a5cce2bb1c42159d0adad5cd0c48ba, role_id=47cde553d7ee465bbe685a05c00b94e2)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:24.062296 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2d92bd-8792-4f94-80e3-a52b374bbc4e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:24.065263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:24.076245 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 561/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:19:23 2022] PUT /identity/v3/projects/db2b557448f04e4aabfb2859cd1be4a1/users/c1a5cce2bb1c42159d0adad5cd0c48ba/roles/47cde553d7ee465bbe685a05c00b94e2 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:19:26.029658 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b429d2-9e08-426d-9f25-1318767734ff None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:26.030115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b429d2-9e08-426d-9f25-1318767734ff None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:26.030215 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b429d2-9e08-426d-9f25-1318767734ff None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:26.031542 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 563/1124] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:19:26.036609 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab8693-52db-4ae5-887a-ab5921c8ac04 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:26.037035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab8693-52db-4ae5-887a-ab5921c8ac04 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:26.037137 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab8693-52db-4ae5-887a-ab5921c8ac04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:26.055103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-daab8693-52db-4ae5-887a-ab5921c8ac04 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:26.056261 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-daab8693-52db-4ae5-887a-ab5921c8ac04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:26.119268 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 562/1125] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:19:26.141892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b429d2-9e08-426d-9f25-1318767734ff None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:26.191670 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:26.241384 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b429d2-9e08-426d-9f25-1318767734ff None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:26.274607 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd01013-0159-4373-8411-d8fdf2f35390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:26.275166 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd01013-0159-4373-8411-d8fdf2f35390 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:26.275444 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd01013-0159-4373-8411-d8fdf2f35390 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:26.275715 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd01013-0159-4373-8411-d8fdf2f35390 None admin] PATH_INFO: `/v3/roles/47cde553d7ee465bbe685a05c00b94e2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:26.277170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd01013-0159-4373-8411-d8fdf2f35390 None admin] RBAC: Authorizing `identity:get_role(role_id=47cde553d7ee465bbe685a05c00b94e2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:26.278521 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd01013-0159-4373-8411-d8fdf2f35390 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:26.279830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 564/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:19:26 2022] GET /identity/v3/roles/47cde553d7ee465bbe685a05c00b94e2 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:26.285455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab8693-52db-4ae5-887a-ab5921c8ac04 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:26.380785 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab8693-52db-4ae5-887a-ab5921c8ac04 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:26.404738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:26.414012 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f02de-c003-4cc9-ac1a-e633bb09e760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:26.414639 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f02de-c003-4cc9-ac1a-e633bb09e760 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:26.414918 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f02de-c003-4cc9-ac1a-e633bb09e760 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:26.415193 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708f02de-c003-4cc9-ac1a-e633bb09e760 None admin] PATH_INFO: `/v3/users/c1a5cce2bb1c42159d0adad5cd0c48ba` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:26.416363 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708f02de-c003-4cc9-ac1a-e633bb09e760 None admin] RBAC: Authorizing `identity:get_user(user_id=c1a5cce2bb1c42159d0adad5cd0c48ba)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:26.417800 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708f02de-c003-4cc9-ac1a-e633bb09e760 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:26.419376 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 563/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:19:26 2022] GET /identity/v3/users/c1a5cce2bb1c42159d0adad5cd0c48ba => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:26.425059 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd01013-0159-4373-8411-d8fdf2f35390 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:26.520934 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd01013-0159-4373-8411-d8fdf2f35390 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:26.537769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:26.554645 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15023bc7-8f7e-4fc7-b5f8-c28b4b9f4d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:26.555246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15023bc7-8f7e-4fc7-b5f8-c28b4b9f4d0a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:26.555525 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15023bc7-8f7e-4fc7-b5f8-c28b4b9f4d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:26.555797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15023bc7-8f7e-4fc7-b5f8-c28b4b9f4d0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:26.619351 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15023bc7-8f7e-4fc7-b5f8-c28b4b9f4d0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:26.620812 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15023bc7-8f7e-4fc7-b5f8-c28b4b9f4d0a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:26.714257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 565/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:19:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 12:19:26.720001 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f02de-c003-4cc9-ac1a-e633bb09e760 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:26.814964 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708f02de-c003-4cc9-ac1a-e633bb09e760 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:26.847040 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6235c-ef2c-435b-9bab-e3207adcda82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:26.847650 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6235c-ef2c-435b-9bab-e3207adcda82 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:26.847929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6235c-ef2c-435b-9bab-e3207adcda82 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:26.848200 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6235c-ef2c-435b-9bab-e3207adcda82 None admin] PATH_INFO: `/v3/projects/db2b557448f04e4aabfb2859cd1be4a1` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:26.849356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de6235c-ef2c-435b-9bab-e3207adcda82 None admin] RBAC: Authorizing `identity:get_project(project_id=db2b557448f04e4aabfb2859cd1be4a1)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:26.851338 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de6235c-ef2c-435b-9bab-e3207adcda82 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:26.852974 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 564/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:19:26 2022] GET /identity/v3/projects/db2b557448f04e4aabfb2859cd1be4a1 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:26.858742 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15023bc7-8f7e-4fc7-b5f8-c28b4b9f4d0a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:26.876514 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:26.953203 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15023bc7-8f7e-4fc7-b5f8-c28b4b9f4d0a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:26.986758 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146f780-4d5a-4e3d-abf1-75ebb695e7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:26.987383 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146f780-4d5a-4e3d-abf1-75ebb695e7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:26.987660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146f780-4d5a-4e3d-abf1-75ebb695e7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:26.987930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146f780-4d5a-4e3d-abf1-75ebb695e7d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:26.988742 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146f780-4d5a-4e3d-abf1-75ebb695e7d9 None admin] RBAC: Adding query filter params (role.id=47cde553d7ee465bbe685a05c00b94e2, scope.project.id=db2b557448f04e4aabfb2859cd1be4a1, user.id=c1a5cce2bb1c42159d0adad5cd0c48ba) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:26.989050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146f780-4d5a-4e3d-abf1-75ebb695e7d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:26.991315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146f780-4d5a-4e3d-abf1-75ebb695e7d9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:26.997393 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 566/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:19:26 2022] GET /identity/v3/role_assignments?user.id=c1a5cce2bb1c42159d0adad5cd0c48ba&scope.project.id=db2b557448f04e4aabfb2859cd1be4a1&role.id=47cde553d7ee465bbe685a05c00b94e2 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:28.978936 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed61483-60d0-4732-8653-d7353da7dafe None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:28.979487 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed61483-60d0-4732-8653-d7353da7dafe None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:28.979487 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed61483-60d0-4732-8653-d7353da7dafe None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:28.980841 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 565/1131] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:19:28.985837 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2b00f-3aea-4789-8d52-1a9e721e50f8 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:28.986216 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2b00f-3aea-4789-8d52-1a9e721e50f8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:28.986384 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2b00f-3aea-4789-8d52-1a9e721e50f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:29.005240 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1ea2b00f-3aea-4789-8d52-1a9e721e50f8 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:29.007076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1ea2b00f-3aea-4789-8d52-1a9e721e50f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:29.071626 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 567/1132] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:19:29.094872 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed61483-60d0-4732-8653-d7353da7dafe None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:29.155975 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:29.194281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed61483-60d0-4732-8653-d7353da7dafe None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:29.227076 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6010a65d-0a6d-4bb7-8343-36fd6ea204bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:29.227697 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010a65d-0a6d-4bb7-8343-36fd6ea204bd None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:29.227975 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010a65d-0a6d-4bb7-8343-36fd6ea204bd None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:29.228246 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010a65d-0a6d-4bb7-8343-36fd6ea204bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:29.229458 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6010a65d-0a6d-4bb7-8343-36fd6ea204bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:29.230833 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6010a65d-0a6d-4bb7-8343-36fd6ea204bd None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:29.232403 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 566/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:19:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:29.238170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2b00f-3aea-4789-8d52-1a9e721e50f8 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:29.252056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:29.333458 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2b00f-3aea-4789-8d52-1a9e721e50f8 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:29.365625 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e89c4-e777-47e6-b7bd-78ca2ecafdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:29.366488 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e89c4-e777-47e6-b7bd-78ca2ecafdba None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:29.366762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e89c4-e777-47e6-b7bd-78ca2ecafdba None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:29.367028 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e89c4-e777-47e6-b7bd-78ca2ecafdba None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:29.367691 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e89c4-e777-47e6-b7bd-78ca2ecafdba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:29.369117 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e89c4-e777-47e6-b7bd-78ca2ecafdba None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:29.394576 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 568/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Aug 6 12:19:29 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:19:31.377281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e60f8-69cc-43e2-9b9d-5c989e97fa6b None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:31.377832 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e60f8-69cc-43e2-9b9d-5c989e97fa6b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:31.377832 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e60f8-69cc-43e2-9b9d-5c989e97fa6b None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:31.379165 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 567/1135] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:19:31.384347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245dd8dd-7a9d-4874-af04-c9636963fb9e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:31.384776 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245dd8dd-7a9d-4874-af04-c9636963fb9e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:31.384907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245dd8dd-7a9d-4874-af04-c9636963fb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:31.403062 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-245dd8dd-7a9d-4874-af04-c9636963fb9e None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:31.404239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-245dd8dd-7a9d-4874-af04-c9636963fb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:31.461158 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:31.470125 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 569/1136] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 12:19:31.495031 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e60f8-69cc-43e2-9b9d-5c989e97fa6b None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:31.593215 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e60f8-69cc-43e2-9b9d-5c989e97fa6b None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:31.620185 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:31.626125 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e154c-bd62-4baf-90d1-9d38d751aca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:31.626747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e154c-bd62-4baf-90d1-9d38d751aca3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:31.627026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e154c-bd62-4baf-90d1-9d38d751aca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:31.627347 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e154c-bd62-4baf-90d1-9d38d751aca3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:31.628465 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e154c-bd62-4baf-90d1-9d38d751aca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:31.629892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e154c-bd62-4baf-90d1-9d38d751aca3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:31.631966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 568/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:19:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:31.638536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245dd8dd-7a9d-4874-af04-c9636963fb9e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:31.734566 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245dd8dd-7a9d-4874-af04-c9636963fb9e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:31.767291 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0072e4-e666-45c5-8691-2a457307283e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:31.767891 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0072e4-e666-45c5-8691-2a457307283e None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:31.768170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0072e4-e666-45c5-8691-2a457307283e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:31.768473 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0072e4-e666-45c5-8691-2a457307283e None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:31.769145 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0072e4-e666-45c5-8691-2a457307283e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:31.770561 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0072e4-e666-45c5-8691-2a457307283e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:31.791531 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 570/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Aug 6 12:19:31 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:19:33.787820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19e8f3-17d8-44ab-a084-7a5e779b31c7 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:33.788398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19e8f3-17d8-44ab-a084-7a5e779b31c7 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:33.788398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19e8f3-17d8-44ab-a084-7a5e779b31c7 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:33.789739 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 569/1139] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:19:33.794717 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a27454-224a-4091-a685-7df29acfa08a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:33.795143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a27454-224a-4091-a685-7df29acfa08a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:33.795250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a27454-224a-4091-a685-7df29acfa08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:33.812907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-35a27454-224a-4091-a685-7df29acfa08a None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:33.814069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-35a27454-224a-4091-a685-7df29acfa08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:33.859319 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:33.881423 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 571/1140] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:19:33.904673 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19e8f3-17d8-44ab-a084-7a5e779b31c7 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:34.000983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19e8f3-17d8-44ab-a084-7a5e779b31c7 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:34.034136 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa1788-0192-489b-aad3-00e46362adc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:34.034755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa1788-0192-489b-aad3-00e46362adc7 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:34.035034 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa1788-0192-489b-aad3-00e46362adc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:34.035307 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa1788-0192-489b-aad3-00e46362adc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:34.045073 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa1788-0192-489b-aad3-00e46362adc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:19:34.049892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-fdaa1788-0192-489b-aad3-00e46362adc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:19:34.049892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:34.049892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:34.049892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:34.049892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:34.049892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:34.049892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:34.049892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:34.049892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:34.049892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:34.049892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:34.049892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:34.049892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:19:34.049892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:19:34.049892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:19:34.049892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:19:34.051038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:19:34.051038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:19:34.051038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:19:34.051038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:19:34.051038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:34.051038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:34.051038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:34.051038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:34.051038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:34.051038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:34.051038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:34.051038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:34.051038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:34.051038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:34.051038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:34.051038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:34.052248 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:34.052248 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:34.052248 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:34.052248 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:34.052248 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:19:34.052248 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:19:34.052248 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:19:34.052248 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:19:34.052248 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:19:34.052248 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:19:34.052248 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:19:34.052248 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:19:34.052248 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 570/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:19:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:19:34.056992 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a27454-224a-4091-a685-7df29acfa08a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:34.151742 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a27454-224a-4091-a685-7df29acfa08a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:34.184797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2b7b8-44d7-4682-81b7-5579d267de1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:34.185410 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2b7b8-44d7-4682-81b7-5579d267de1e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:34.185695 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2b7b8-44d7-4682-81b7-5579d267de1e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:34.185975 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2b7b8-44d7-4682-81b7-5579d267de1e None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:34.186679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e2b7b8-44d7-4682-81b7-5579d267de1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:34.186976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e2b7b8-44d7-4682-81b7-5579d267de1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:34.188338 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e2b7b8-44d7-4682-81b7-5579d267de1e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:34.196104 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 572/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:19:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:34.201693 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa1788-0192-489b-aad3-00e46362adc7 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:34.259205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:34.299337 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa1788-0192-489b-aad3-00e46362adc7 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:34.332135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae8023-953e-425f-aa56-fa7f5d219456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:34.332777 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae8023-953e-425f-aa56-fa7f5d219456 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:34.333053 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae8023-953e-425f-aa56-fa7f5d219456 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:34.333322 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae8023-953e-425f-aa56-fa7f5d219456 None admin] PATH_INFO: `/v3/users/49dc7d93a461476db992c4e39e9ebad5` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:34.345750 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae8023-953e-425f-aa56-fa7f5d219456 None admin] RBAC: Authorizing `identity:get_user(user_id=49dc7d93a461476db992c4e39e9ebad5)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:34.347173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae8023-953e-425f-aa56-fa7f5d219456 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:34.348790 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 571/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:19:34 2022] GET /identity/v3/users/49dc7d93a461476db992c4e39e9ebad5 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:34.355558 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2b7b8-44d7-4682-81b7-5579d267de1e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:34.399959 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:34.455060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2b7b8-44d7-4682-81b7-5579d267de1e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:34.488490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94541f8b-874f-4aa7-8fdf-391c697b3677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:34.489090 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94541f8b-874f-4aa7-8fdf-391c697b3677 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:34.489364 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94541f8b-874f-4aa7-8fdf-391c697b3677 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:34.489630 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94541f8b-874f-4aa7-8fdf-391c697b3677 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:34.554015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94541f8b-874f-4aa7-8fdf-391c697b3677 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:34.555375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94541f8b-874f-4aa7-8fdf-391c697b3677 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:34.594312 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:34.652733 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 573/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:19:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 12:19:34.658586 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae8023-953e-425f-aa56-fa7f5d219456 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:34.754795 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae8023-953e-425f-aa56-fa7f5d219456 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:34.771798 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:34.789462 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5dfe7-9618-4035-9d06-48cd5a2ea277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:34.790082 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5dfe7-9618-4035-9d06-48cd5a2ea277 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:34.790357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5dfe7-9618-4035-9d06-48cd5a2ea277 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:34.790626 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5dfe7-9618-4035-9d06-48cd5a2ea277 None admin] PATH_INFO: `/v3/projects/e0d0d68a16c9436cbc517c44b6b479b6` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:34.791756 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5dfe7-9618-4035-9d06-48cd5a2ea277 None admin] RBAC: Authorizing `identity:get_project(project_id=e0d0d68a16c9436cbc517c44b6b479b6)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:34.793609 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5dfe7-9618-4035-9d06-48cd5a2ea277 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:34.795240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 572/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:19:34 2022] GET /identity/v3/projects/e0d0d68a16c9436cbc517c44b6b479b6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:34.801320 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94541f8b-874f-4aa7-8fdf-391c697b3677 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:34.897152 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94541f8b-874f-4aa7-8fdf-391c697b3677 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:34.929381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba59014-733c-415f-afd5-8b8f81922952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:34.929970 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba59014-733c-415f-afd5-8b8f81922952 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:34.930246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba59014-733c-415f-afd5-8b8f81922952 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:34.930516 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba59014-733c-415f-afd5-8b8f81922952 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:34.931284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba59014-733c-415f-afd5-8b8f81922952 None admin] RBAC: Adding query filter params (role.id=144c8425688b409384e1e8ed8279d88e, scope.project.id=e0d0d68a16c9436cbc517c44b6b479b6, user.id=49dc7d93a461476db992c4e39e9ebad5) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:34.931586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba59014-733c-415f-afd5-8b8f81922952 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:34.933011 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba59014-733c-415f-afd5-8b8f81922952 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:34.938155 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 574/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:19:34 2022] GET /identity/v3/role_assignments?user.id=49dc7d93a461476db992c4e39e9ebad5&scope.project.id=e0d0d68a16c9436cbc517c44b6b479b6&role.id=144c8425688b409384e1e8ed8279d88e => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:36.896900 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9c116-2c25-4dfa-8d7d-5f555430231e None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:36.897450 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9c116-2c25-4dfa-8d7d-5f555430231e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:36.897450 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9c116-2c25-4dfa-8d7d-5f555430231e None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:36.898757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 573/1147] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:19:36.903961 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58cf37-df78-4de9-a7fe-99c19efab9bc None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:36.904345 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58cf37-df78-4de9-a7fe-99c19efab9bc None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:36.904537 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58cf37-df78-4de9-a7fe-99c19efab9bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:36.917732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:36.922938 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6e58cf37-df78-4de9-a7fe-99c19efab9bc None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:36.924114 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6e58cf37-df78-4de9-a7fe-99c19efab9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:36.989679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 575/1148] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:19:37.012255 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9c116-2c25-4dfa-8d7d-5f555430231e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:37.110144 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9c116-2c25-4dfa-8d7d-5f555430231e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:37.143020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0497bb9a-aa03-428c-a9ca-d610b9724217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:37.143637 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497bb9a-aa03-428c-a9ca-d610b9724217 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:37.143919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497bb9a-aa03-428c-a9ca-d610b9724217 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:37.144195 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497bb9a-aa03-428c-a9ca-d610b9724217 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:37.154612 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0497bb9a-aa03-428c-a9ca-d610b9724217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:19:37.159509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-0497bb9a-aa03-428c-a9ca-d610b9724217 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:19:37.159509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:37.159509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:37.159509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:37.159509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:37.159509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:37.159509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:37.159509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:37.159509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:37.159509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:37.159509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:37.159509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:37.159509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:19:37.159509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:19:37.159509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:19:37.159509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:19:37.160602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:19:37.160602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:19:37.160602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:19:37.160602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:19:37.160602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:37.160602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:37.160602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:37.160602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:37.160602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:37.160602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:37.160602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:37.160602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:37.160602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:37.160602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:37.160602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:37.160602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:37.161684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:37.161684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:37.161684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:37.161684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:37.161684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:19:37.161684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:19:37.161684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:19:37.161684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:19:37.161684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:19:37.161684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:19:37.161684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:19:37.161684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:19:37.161684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 574/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:19:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:19:37.167108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58cf37-df78-4de9-a7fe-99c19efab9bc None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:37.261510 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58cf37-df78-4de9-a7fe-99c19efab9bc None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:37.294578 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1634ba-bc7b-4085-8c56-aac5c3bef06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:37.295176 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1634ba-bc7b-4085-8c56-aac5c3bef06f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:37.295456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1634ba-bc7b-4085-8c56-aac5c3bef06f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:37.295728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1634ba-bc7b-4085-8c56-aac5c3bef06f None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:37.296467 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1634ba-bc7b-4085-8c56-aac5c3bef06f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:37.296767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1634ba-bc7b-4085-8c56-aac5c3bef06f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:37.298114 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1634ba-bc7b-4085-8c56-aac5c3bef06f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:37.300213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:37.306693 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 576/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:19:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:19:37.312368 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0497bb9a-aa03-428c-a9ca-d610b9724217 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:37.357517 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:37.409666 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0497bb9a-aa03-428c-a9ca-d610b9724217 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:37.442025 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd303620-60fe-4b64-a80f-03bffe274be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:37.442643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd303620-60fe-4b64-a80f-03bffe274be2 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:37.442952 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd303620-60fe-4b64-a80f-03bffe274be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:37.443228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd303620-60fe-4b64-a80f-03bffe274be2 None admin] PATH_INFO: `/v3/users/49dc7d93a461476db992c4e39e9ebad5` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:37.444409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd303620-60fe-4b64-a80f-03bffe274be2 None admin] RBAC: Authorizing `identity:get_user(user_id=49dc7d93a461476db992c4e39e9ebad5)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:37.445839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd303620-60fe-4b64-a80f-03bffe274be2 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:37.447392 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 575/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:19:37 2022] GET /identity/v3/users/49dc7d93a461476db992c4e39e9ebad5 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:37.452980 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1634ba-bc7b-4085-8c56-aac5c3bef06f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:37.554219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1634ba-bc7b-4085-8c56-aac5c3bef06f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:37.587189 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb7f30-047d-4987-a520-7b9f18507980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:37.587813 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb7f30-047d-4987-a520-7b9f18507980 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:37.588091 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb7f30-047d-4987-a520-7b9f18507980 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:37.588398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb7f30-047d-4987-a520-7b9f18507980 None admin] PATH_INFO: `/v3/projects/e0d0d68a16c9436cbc517c44b6b479b6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:37.589540 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb7f30-047d-4987-a520-7b9f18507980 None admin] RBAC: Authorizing `identity:get_project(project_id=e0d0d68a16c9436cbc517c44b6b479b6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:37.590957 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb7f30-047d-4987-a520-7b9f18507980 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:37.592643 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 577/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:19:37 2022] GET /identity/v3/projects/e0d0d68a16c9436cbc517c44b6b479b6 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:37.598503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd303620-60fe-4b64-a80f-03bffe274be2 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:37.693651 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd303620-60fe-4b64-a80f-03bffe274be2 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:37.705867 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:37.726534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520be31-b416-4b3c-860d-f16affa777e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:37.727183 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520be31-b416-4b3c-860d-f16affa777e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:37.727463 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520be31-b416-4b3c-860d-f16affa777e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:37.727737 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520be31-b416-4b3c-860d-f16affa777e5 None admin] PATH_INFO: `/v3/projects/e0d0d68a16c9436cbc517c44b6b479b6/users/49dc7d93a461476db992c4e39e9ebad5/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:37.729672 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520be31-b416-4b3c-860d-f16affa777e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=e0d0d68a16c9436cbc517c44b6b479b6, user_id=49dc7d93a461476db992c4e39e9ebad5, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:37.731099 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520be31-b416-4b3c-860d-f16affa777e5 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:37.744284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 576/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:19:37 2022] PUT /identity/v3/projects/e0d0d68a16c9436cbc517c44b6b479b6/users/49dc7d93a461476db992c4e39e9ebad5/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:19:39.717459 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c583a-e183-4613-97cb-01e2f62bab35 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:39.718010 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c583a-e183-4613-97cb-01e2f62bab35 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:39.718010 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681c583a-e183-4613-97cb-01e2f62bab35 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:39.719332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 578/1154] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:19:39.724434 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d2e6f-23e2-4bc8-a6fc-e74fee3c78cd None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:39.724847 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d2e6f-23e2-4bc8-a6fc-e74fee3c78cd None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:39.724970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d2e6f-23e2-4bc8-a6fc-e74fee3c78cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:39.743723 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c71d2e6f-23e2-4bc8-a6fc-e74fee3c78cd None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:39.744915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c71d2e6f-23e2-4bc8-a6fc-e74fee3c78cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:39.808637 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 577/1155] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:19:39.830995 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c583a-e183-4613-97cb-01e2f62bab35 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:39.896118 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:39.932507 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681c583a-e183-4613-97cb-01e2f62bab35 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:39.964417 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f6886-7281-41b6-bb19-718c01df8dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:39.965037 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f6886-7281-41b6-bb19-718c01df8dee None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:39.965311 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f6886-7281-41b6-bb19-718c01df8dee None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:39.965580 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f6886-7281-41b6-bb19-718c01df8dee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:39.975883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f6886-7281-41b6-bb19-718c01df8dee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:19:39.980687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-119f6886-7281-41b6-bb19-718c01df8dee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:19:39.980687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:39.980687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:39.980687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:39.980687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:39.980687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:39.980687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:39.980687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:39.980687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:39.980687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:39.980687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:39.980687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:39.980687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:19:39.980687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:19:39.980687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:19:39.980687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:19:39.981583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:19:39.981583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:19:39.981583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:19:39.981583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:19:39.981583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:39.981583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:39.981583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:39.981583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:39.981583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:39.981583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:39.981583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:39.981583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:39.981583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:39.981583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:39.981583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:39.981583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:39.982433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:39.982433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:39.982433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:39.982433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:39.982433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:19:39.982433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:19:39.982433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:19:39.982433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:19:39.982433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:19:39.982433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:19:39.982433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:19:39.982433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:19:39.982433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 579/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:19:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:19:39.987898 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d2e6f-23e2-4bc8-a6fc-e74fee3c78cd None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:40.082937 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d2e6f-23e2-4bc8-a6fc-e74fee3c78cd None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:40.084437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:40.118010 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b05f6-0ecd-4826-90db-61703bda899b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:40.118613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b05f6-0ecd-4826-90db-61703bda899b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:40.118889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b05f6-0ecd-4826-90db-61703bda899b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:40.119159 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b05f6-0ecd-4826-90db-61703bda899b None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:40.119865 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b05f6-0ecd-4826-90db-61703bda899b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:40.120164 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b05f6-0ecd-4826-90db-61703bda899b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:40.121548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b05f6-0ecd-4826-90db-61703bda899b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:40.129221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 578/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:19:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:40.134976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f6886-7281-41b6-bb19-718c01df8dee None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:40.236398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f6886-7281-41b6-bb19-718c01df8dee None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:40.260916 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:40.269855 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a762f-c392-475a-9dbd-6d7be6ca4202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:40.270478 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a762f-c392-475a-9dbd-6d7be6ca4202 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:40.270754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a762f-c392-475a-9dbd-6d7be6ca4202 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:40.271616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a762f-c392-475a-9dbd-6d7be6ca4202 None admin] PATH_INFO: `/v3/users/49dc7d93a461476db992c4e39e9ebad5` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:40.272715 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885a762f-c392-475a-9dbd-6d7be6ca4202 None admin] RBAC: Authorizing `identity:get_user(user_id=49dc7d93a461476db992c4e39e9ebad5)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:40.274123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885a762f-c392-475a-9dbd-6d7be6ca4202 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:40.275699 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 580/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:19:40 2022] GET /identity/v3/users/49dc7d93a461476db992c4e39e9ebad5 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:40.281487 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b05f6-0ecd-4826-90db-61703bda899b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:40.376939 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b05f6-0ecd-4826-90db-61703bda899b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:40.408529 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50aa14-706e-4a00-ad89-d377f4f561a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:40.409117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50aa14-706e-4a00-ad89-d377f4f561a6 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:40.409391 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50aa14-706e-4a00-ad89-d377f4f561a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:40.409661 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50aa14-706e-4a00-ad89-d377f4f561a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:40.438342 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:40.473556 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50aa14-706e-4a00-ad89-d377f4f561a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:40.475558 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50aa14-706e-4a00-ad89-d377f4f561a6 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:40.572030 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 579/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:19:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 12:19:40.577986 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a762f-c392-475a-9dbd-6d7be6ca4202 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:40.675083 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a762f-c392-475a-9dbd-6d7be6ca4202 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:40.707762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6156c-b9c6-418a-abb9-07aeba277345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:40.708942 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6156c-b9c6-418a-abb9-07aeba277345 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:40.709225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6156c-b9c6-418a-abb9-07aeba277345 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:40.709499 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6156c-b9c6-418a-abb9-07aeba277345 None admin] PATH_INFO: `/v3/projects/e0d0d68a16c9436cbc517c44b6b479b6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:40.710644 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6156c-b9c6-418a-abb9-07aeba277345 None admin] RBAC: Authorizing `identity:get_project(project_id=e0d0d68a16c9436cbc517c44b6b479b6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:40.712065 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6156c-b9c6-418a-abb9-07aeba277345 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:40.713728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 581/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:19:40 2022] GET /identity/v3/projects/e0d0d68a16c9436cbc517c44b6b479b6 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:40.719732 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50aa14-706e-4a00-ad89-d377f4f561a6 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:40.778893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:40.821089 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50aa14-706e-4a00-ad89-d377f4f561a6 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:40.853825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b4b9d-30f9-4db5-b6c2-e94d56007239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:40.854360 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b4b9d-30f9-4db5-b6c2-e94d56007239 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:40.854639 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b4b9d-30f9-4db5-b6c2-e94d56007239 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:40.854910 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b4b9d-30f9-4db5-b6c2-e94d56007239 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:40.855684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405b4b9d-30f9-4db5-b6c2-e94d56007239 None admin] RBAC: Adding query filter params (role.id=144c8425688b409384e1e8ed8279d88e, scope.project.id=e0d0d68a16c9436cbc517c44b6b479b6, user.id=49dc7d93a461476db992c4e39e9ebad5) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:40.855990 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405b4b9d-30f9-4db5-b6c2-e94d56007239 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:40.857444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405b4b9d-30f9-4db5-b6c2-e94d56007239 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:40.863025 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 580/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:19:40 2022] GET /identity/v3/role_assignments?user.id=49dc7d93a461476db992c4e39e9ebad5&scope.project.id=e0d0d68a16c9436cbc517c44b6b479b6&role.id=144c8425688b409384e1e8ed8279d88e => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:42.845660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb160fd-1254-49a5-8423-6045a54591bf None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:42.846186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb160fd-1254-49a5-8423-6045a54591bf None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:42.846186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb160fd-1254-49a5-8423-6045a54591bf None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:42.847550 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 582/1162] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:19:42.852943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f2a70-81a4-49f7-a366-5a60add11eca None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:42.853323 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f2a70-81a4-49f7-a366-5a60add11eca None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:42.853507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f2a70-81a4-49f7-a366-5a60add11eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:42.871239 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3e2f2a70-81a4-49f7-a366-5a60add11eca None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:42.872436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3e2f2a70-81a4-49f7-a366-5a60add11eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:42.935840 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 581/1163] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:19:42.957931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb160fd-1254-49a5-8423-6045a54591bf None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:43.021627 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:43.060048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb160fd-1254-49a5-8423-6045a54591bf None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:43.093323 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46851800-a560-4831-9239-38b8c36b8067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:43.093925 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46851800-a560-4831-9239-38b8c36b8067 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:43.094204 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46851800-a560-4831-9239-38b8c36b8067 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:43.094480 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46851800-a560-4831-9239-38b8c36b8067 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:43.160064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46851800-a560-4831-9239-38b8c36b8067 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:43.161469 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46851800-a560-4831-9239-38b8c36b8067 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:43.218166 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:43.257182 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 583/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:19:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 12:19:43.263170 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f2a70-81a4-49f7-a366-5a60add11eca None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:43.317384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:43.356676 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f2a70-81a4-49f7-a366-5a60add11eca None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:43.389302 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10af57-6010-4e7e-9568-6c5854ece81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:43.389916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10af57-6010-4e7e-9568-6c5854ece81c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:43.390190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10af57-6010-4e7e-9568-6c5854ece81c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:43.390459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10af57-6010-4e7e-9568-6c5854ece81c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:43.397113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10af57-6010-4e7e-9568-6c5854ece81c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:43.398527 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10af57-6010-4e7e-9568-6c5854ece81c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:43.409629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-9e10af57-6010-4e7e-9568-6c5854ece81c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 06 12:19:43.409629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:43.409629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 06 12:19:43.409629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 06 12:19:43.409629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:19:43.409629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:19:43.409629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:19:43.409629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:19:43.409629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 06 12:19:43.409629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:19:43.409629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:19:43.409629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:19:43.409629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:43.409629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:43.409629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:43.409629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:43.409629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:43.410787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:43.410787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:43.410787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:43.410787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:43.410787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:43.410787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:43.410787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:19:43.410787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:19:43.410787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:19:43.410787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:19:43.410787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:43.410787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:43.410787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:43.410787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:43.410787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:43.410787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:43.411782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:43.411782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:43.411782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:43.411782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:43.411782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:43.411782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:43.411782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:43.411782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:43.411782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:43.411782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:43.411782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 06 12:19:43.411782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:19:43.411782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 06 12:19:43.411782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:19:43.411782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 582/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Sat Aug 6 12:19:43 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:19:43.416600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46851800-a560-4831-9239-38b8c36b8067 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:43.511150 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46851800-a560-4831-9239-38b8c36b8067 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:43.543643 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8ad1d-f5c5-455a-8920-4dfebd84801c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:43.544243 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8ad1d-f5c5-455a-8920-4dfebd84801c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:43.544552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8ad1d-f5c5-455a-8920-4dfebd84801c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:43.544828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8ad1d-f5c5-455a-8920-4dfebd84801c None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:43.545512 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8ad1d-f5c5-455a-8920-4dfebd84801c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:43.545815 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8ad1d-f5c5-455a-8920-4dfebd84801c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:43.547161 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8ad1d-f5c5-455a-8920-4dfebd84801c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:43.554068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 584/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Sat Aug 6 12:19:43 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:45.562320 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e617eb-b4dc-4f78-9fdd-60407f3a1f10 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:45.562840 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e617eb-b4dc-4f78-9fdd-60407f3a1f10 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:45.562840 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e617eb-b4dc-4f78-9fdd-60407f3a1f10 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:45.564229 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 583/1167] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:19:45.569483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f0b64-3f82-4110-a8c8-c5e12635d2b8 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:45.569912 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f0b64-3f82-4110-a8c8-c5e12635d2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:45.570045 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f0b64-3f82-4110-a8c8-c5e12635d2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:45.588266 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-005f0b64-3f82-4110-a8c8-c5e12635d2b8 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:45.589477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-005f0b64-3f82-4110-a8c8-c5e12635d2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:45.612126 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:45.657544 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 585/1168] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:19:45.680259 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e617eb-b4dc-4f78-9fdd-60407f3a1f10 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:45.779660 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e617eb-b4dc-4f78-9fdd-60407f3a1f10 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:45.797091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:45.813327 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a26e3-5aff-4e16-8a2e-3fd58d871d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:45.813874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a26e3-5aff-4e16-8a2e-3fd58d871d98 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:45.814154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a26e3-5aff-4e16-8a2e-3fd58d871d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:45.814427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a26e3-5aff-4e16-8a2e-3fd58d871d98 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:45.815137 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a26e3-5aff-4e16-8a2e-3fd58d871d98 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:45.816530 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a26e3-5aff-4e16-8a2e-3fd58d871d98 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:45.832063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 584/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Sat Aug 6 12:19:45 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:19:47.841671 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f03bbe-e017-415c-88a4-0f6c288fc0bc None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:47.842225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f03bbe-e017-415c-88a4-0f6c288fc0bc None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:47.842225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f03bbe-e017-415c-88a4-0f6c288fc0bc None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:47.843556 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 586/1170] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:19:47.848603 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d6193-1133-47cf-aca2-e44a0a60b828 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:47.849012 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d6193-1133-47cf-aca2-e44a0a60b828 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:47.849150 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d6193-1133-47cf-aca2-e44a0a60b828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:47.867686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-2a7d6193-1133-47cf-aca2-e44a0a60b828 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:47.868895 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2a7d6193-1133-47cf-aca2-e44a0a60b828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:47.933810 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 585/1171] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:19:47.961898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f03bbe-e017-415c-88a4-0f6c288fc0bc None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:48.060283 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f03bbe-e017-415c-88a4-0f6c288fc0bc None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:48.091914 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332a263-9ff0-49bd-a91d-f30adbbdfac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:48.092501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332a263-9ff0-49bd-a91d-f30adbbdfac7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:48.092787 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332a263-9ff0-49bd-a91d-f30adbbdfac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:48.093049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332a263-9ff0-49bd-a91d-f30adbbdfac7 None admin] PATH_INFO: `/v3/domains/f77ff2b3fc8d49fab74e2d531cf41660` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:48.103501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332a263-9ff0-49bd-a91d-f30adbbdfac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f77ff2b3fc8d49fab74e2d531cf41660)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:48.104952 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332a263-9ff0-49bd-a91d-f30adbbdfac7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:48.106521 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 587/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Sat Aug 6 12:19:47 2022] GET /identity/v3/domains/f77ff2b3fc8d49fab74e2d531cf41660 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:48.112322 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d6193-1133-47cf-aca2-e44a0a60b828 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:48.205673 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d6193-1133-47cf-aca2-e44a0a60b828 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:48.214453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:48.240851 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f07a2-5a06-437d-95d6-b7c15ef75ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:48.241462 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f07a2-5a06-437d-95d6-b7c15ef75ac3 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:48.241738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f07a2-5a06-437d-95d6-b7c15ef75ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:48.242011 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f07a2-5a06-437d-95d6-b7c15ef75ac3 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:48.242699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f07a2-5a06-437d-95d6-b7c15ef75ac3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:48.244127 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f07a2-5a06-437d-95d6-b7c15ef75ac3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:48.269431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 586/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Aug 6 12:19:48 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:19:50.268573 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff644f7-ef22-441f-962d-e55d96ffdbd6 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:50.269084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff644f7-ef22-441f-962d-e55d96ffdbd6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:50.269084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff644f7-ef22-441f-962d-e55d96ffdbd6 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:50.270397 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 588/1174] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:19:50.280280 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad5cd7-8d62-427b-abe9-d4561a06645c None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:50.280845 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad5cd7-8d62-427b-abe9-d4561a06645c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:50.281355 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad5cd7-8d62-427b-abe9-d4561a06645c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:50.301290 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-b8ad5cd7-8d62-427b-abe9-d4561a06645c None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:50.302523 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b8ad5cd7-8d62-427b-abe9-d4561a06645c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:50.367715 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 587/1175] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:19:50.391910 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff644f7-ef22-441f-962d-e55d96ffdbd6 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:50.409796 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:50.491076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff644f7-ef22-441f-962d-e55d96ffdbd6 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:50.522986 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a039b768-a57e-4ae7-8fab-7e6dfaf17ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:50.523538 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039b768-a57e-4ae7-8fab-7e6dfaf17ebc None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:50.523814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039b768-a57e-4ae7-8fab-7e6dfaf17ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:50.524086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039b768-a57e-4ae7-8fab-7e6dfaf17ebc None admin] PATH_INFO: `/v3/domains/f77ff2b3fc8d49fab74e2d531cf41660` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:50.525241 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039b768-a57e-4ae7-8fab-7e6dfaf17ebc None admin] RBAC: Authorizing `identity:get_domain(domain_id=f77ff2b3fc8d49fab74e2d531cf41660)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:50.526649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039b768-a57e-4ae7-8fab-7e6dfaf17ebc None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:50.528186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 589/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Sat Aug 6 12:19:50 2022] GET /identity/v3/domains/f77ff2b3fc8d49fab74e2d531cf41660 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:50.533682 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad5cd7-8d62-427b-abe9-d4561a06645c None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:50.623775 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:50.634314 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad5cd7-8d62-427b-abe9-d4561a06645c None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:50.667288 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfeab9c-8410-4c02-a028-531a10050ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:50.667911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfeab9c-8410-4c02-a028-531a10050ab1 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:50.668184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfeab9c-8410-4c02-a028-531a10050ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:50.668481 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfeab9c-8410-4c02-a028-531a10050ab1 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:50.669141 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfeab9c-8410-4c02-a028-531a10050ab1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:50.670542 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfeab9c-8410-4c02-a028-531a10050ab1 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:50.690612 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 588/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Aug 6 12:19:50 2022] POST /identity/v3/users => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:19:52.729131 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4218cdd-a340-47b3-a280-8859e1b8a908 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:52.729656 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4218cdd-a340-47b3-a280-8859e1b8a908 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:52.729656 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4218cdd-a340-47b3-a280-8859e1b8a908 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:52.730999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 590/1178] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:19:52.736014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe7e1f-81c6-47a3-baa8-2ad396a47997 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:52.736428 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe7e1f-81c6-47a3-baa8-2ad396a47997 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:52.736604 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe7e1f-81c6-47a3-baa8-2ad396a47997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:52.755138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-7dfe7e1f-81c6-47a3-baa8-2ad396a47997 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:52.756456 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7dfe7e1f-81c6-47a3-baa8-2ad396a47997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:52.820866 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 589/1179] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:19:52.843707 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4218cdd-a340-47b3-a280-8859e1b8a908 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:52.925846 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:52.941987 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4218cdd-a340-47b3-a280-8859e1b8a908 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:52.975138 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e8b0b-2c1a-4375-bad6-0ab05fdcc19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:52.975760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e8b0b-2c1a-4375-bad6-0ab05fdcc19c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:52.976040 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e8b0b-2c1a-4375-bad6-0ab05fdcc19c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:52.976331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e8b0b-2c1a-4375-bad6-0ab05fdcc19c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:52.986143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e8b0b-2c1a-4375-bad6-0ab05fdcc19c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:19:52.991230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-ea0e8b0b-2c1a-4375-bad6-0ab05fdcc19c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:19:52.991230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:52.991230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:52.991230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:52.991230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:52.991230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:52.991230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:52.991230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:52.991230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:52.991230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:52.991230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:52.991230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:52.991230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:19:52.991230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:19:52.991230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:19:52.991230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:19:52.992349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:19:52.992349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:19:52.992349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:19:52.992349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:19:52.992349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:52.992349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:52.992349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:52.992349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:52.992349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:52.992349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:52.992349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:52.992349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:52.992349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:52.992349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:52.992349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:52.992349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:52.993227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:52.993227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:52.993227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:52.993227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:52.993227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:19:52.993227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:19:52.993227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:19:52.993227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:19:52.993227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:19:52.993227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:19:52.993227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:19:52.993227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:19:52.993227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 591/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:19:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:19:52.998731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe7e1f-81c6-47a3-baa8-2ad396a47997 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:53.071498 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:53.094912 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe7e1f-81c6-47a3-baa8-2ad396a47997 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:53.128042 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896c890-829b-49e6-9a39-8ba5e9a3b47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:53.128672 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896c890-829b-49e6-9a39-8ba5e9a3b47a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:53.128947 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896c890-829b-49e6-9a39-8ba5e9a3b47a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:53.129213 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896c890-829b-49e6-9a39-8ba5e9a3b47a None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:53.129939 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d896c890-829b-49e6-9a39-8ba5e9a3b47a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:53.130236 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d896c890-829b-49e6-9a39-8ba5e9a3b47a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:53.131585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d896c890-829b-49e6-9a39-8ba5e9a3b47a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:53.139226 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 590/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:19:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:53.144936 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e8b0b-2c1a-4375-bad6-0ab05fdcc19c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:53.242482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e8b0b-2c1a-4375-bad6-0ab05fdcc19c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:53.276112 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30a207-54ad-4ed3-913f-b67db946a783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:53.276780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30a207-54ad-4ed3-913f-b67db946a783 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:53.277062 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30a207-54ad-4ed3-913f-b67db946a783 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:53.277336 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30a207-54ad-4ed3-913f-b67db946a783 None admin] PATH_INFO: `/v3/users/3a80899bbb324477ab57749f6b704696` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:53.290548 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab30a207-54ad-4ed3-913f-b67db946a783 None admin] RBAC: Authorizing `identity:get_user(user_id=3a80899bbb324477ab57749f6b704696)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:53.291974 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab30a207-54ad-4ed3-913f-b67db946a783 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:53.293648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 592/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:19:53 2022] GET /identity/v3/users/3a80899bbb324477ab57749f6b704696 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:53.299472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896c890-829b-49e6-9a39-8ba5e9a3b47a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:53.394769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896c890-829b-49e6-9a39-8ba5e9a3b47a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:53.426650 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11976f0f-0c8a-4ffc-8d6f-9745caebdb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:53.427252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11976f0f-0c8a-4ffc-8d6f-9745caebdb3b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:53.427530 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11976f0f-0c8a-4ffc-8d6f-9745caebdb3b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:53.427801 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11976f0f-0c8a-4ffc-8d6f-9745caebdb3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:53.435806 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:53.492786 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11976f0f-0c8a-4ffc-8d6f-9745caebdb3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:53.494163 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11976f0f-0c8a-4ffc-8d6f-9745caebdb3b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:53.586260 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 591/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:19:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 12:19:53.592498 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30a207-54ad-4ed3-913f-b67db946a783 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:53.595409 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:53.698092 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30a207-54ad-4ed3-913f-b67db946a783 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:53.731406 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2057c7-52a8-44e9-b20a-5be4e6a4c9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:53.732030 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2057c7-52a8-44e9-b20a-5be4e6a4c9ed None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:53.732322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2057c7-52a8-44e9-b20a-5be4e6a4c9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:53.732610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2057c7-52a8-44e9-b20a-5be4e6a4c9ed None admin] PATH_INFO: `/v3/projects/6935c5cdbbaa4df09abbb28eba39d724` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:53.733749 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2057c7-52a8-44e9-b20a-5be4e6a4c9ed None admin] RBAC: Authorizing `identity:get_project(project_id=6935c5cdbbaa4df09abbb28eba39d724)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:53.735167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2057c7-52a8-44e9-b20a-5be4e6a4c9ed None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:53.736854 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 593/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:19:53 2022] GET /identity/v3/projects/6935c5cdbbaa4df09abbb28eba39d724 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:53.743349 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11976f0f-0c8a-4ffc-8d6f-9745caebdb3b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:53.782328 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:53.840710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11976f0f-0c8a-4ffc-8d6f-9745caebdb3b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:53.872965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74678b-a3d5-4b7c-acea-bdfb640502b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:53.873560 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74678b-a3d5-4b7c-acea-bdfb640502b8 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:53.873835 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74678b-a3d5-4b7c-acea-bdfb640502b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:53.874102 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74678b-a3d5-4b7c-acea-bdfb640502b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:53.874868 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74678b-a3d5-4b7c-acea-bdfb640502b8 None admin] RBAC: Adding query filter params (role.id=144c8425688b409384e1e8ed8279d88e, scope.project.id=6935c5cdbbaa4df09abbb28eba39d724, user.id=3a80899bbb324477ab57749f6b704696) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:53.875168 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74678b-a3d5-4b7c-acea-bdfb640502b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:53.876605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74678b-a3d5-4b7c-acea-bdfb640502b8 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:53.881769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 592/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:19:53 2022] GET /identity/v3/role_assignments?user.id=3a80899bbb324477ab57749f6b704696&scope.project.id=6935c5cdbbaa4df09abbb28eba39d724&role.id=144c8425688b409384e1e8ed8279d88e => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:55.870239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d498fe-0d2e-4896-afed-0dede31cc1e4 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:55.870770 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d498fe-0d2e-4896-afed-0dede31cc1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:55.870770 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d498fe-0d2e-4896-afed-0dede31cc1e4 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:55.872130 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 594/1186] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:19:55.877267 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df15a6a-3fd6-4d8d-ab1d-47d9dd8dd50c None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:55.877648 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df15a6a-3fd6-4d8d-ab1d-47d9dd8dd50c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:55.877813 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df15a6a-3fd6-4d8d-ab1d-47d9dd8dd50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:55.895682 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-1df15a6a-3fd6-4d8d-ab1d-47d9dd8dd50c None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:55.896870 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-1df15a6a-3fd6-4d8d-ab1d-47d9dd8dd50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:55.962587 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 593/1187] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:19:55.985503 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d498fe-0d2e-4896-afed-0dede31cc1e4 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:56.057118 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:56.087473 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d498fe-0d2e-4896-afed-0dede31cc1e4 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:56.120018 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f396ef-1d4e-4b47-9a76-5e80ba0b9026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:56.120716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f396ef-1d4e-4b47-9a76-5e80ba0b9026 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:56.120941 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f396ef-1d4e-4b47-9a76-5e80ba0b9026 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:56.121209 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f396ef-1d4e-4b47-9a76-5e80ba0b9026 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:56.130855 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f396ef-1d4e-4b47-9a76-5e80ba0b9026 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:19:56.135723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-74f396ef-1d4e-4b47-9a76-5e80ba0b9026 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:19:56.135723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:56.135723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:56.135723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:56.135723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:56.135723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:56.135723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:56.135723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:56.135723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:56.135723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:56.135723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:56.135723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:56.135723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:19:56.135723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:19:56.135723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:19:56.135723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:19:56.136809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:19:56.136809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:19:56.136809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:19:56.136809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:19:56.136809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:56.136809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:56.136809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:56.136809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:56.136809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:56.136809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:56.136809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:56.136809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:56.136809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:56.136809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:56.136809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:56.136809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:56.137879 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:56.137879 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:56.137879 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:56.137879 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:56.137879 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:19:56.137879 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:19:56.137879 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:19:56.137879 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:19:56.137879 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:19:56.137879 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:19:56.137879 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:19:56.137879 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:19:56.137879 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 595/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:19:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:19:56.143153 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df15a6a-3fd6-4d8d-ab1d-47d9dd8dd50c None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:56.173080 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:56.238489 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df15a6a-3fd6-4d8d-ab1d-47d9dd8dd50c None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:56.287350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92822888-e8c0-4715-95d9-621d3def5243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:56.288251 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92822888-e8c0-4715-95d9-621d3def5243 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:56.288713 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92822888-e8c0-4715-95d9-621d3def5243 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:56.289130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92822888-e8c0-4715-95d9-621d3def5243 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:56.290236 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92822888-e8c0-4715-95d9-621d3def5243 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:56.290708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92822888-e8c0-4715-95d9-621d3def5243 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:56.292940 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92822888-e8c0-4715-95d9-621d3def5243 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:56.304730 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 594/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:19:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:56.310647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f396ef-1d4e-4b47-9a76-5e80ba0b9026 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:56.407720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f396ef-1d4e-4b47-9a76-5e80ba0b9026 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:56.435223 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:56.441015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce95d76-b94d-4926-b8b5-c886b012854f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:56.441629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce95d76-b94d-4926-b8b5-c886b012854f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:56.442406 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce95d76-b94d-4926-b8b5-c886b012854f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:56.442678 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce95d76-b94d-4926-b8b5-c886b012854f None admin] PATH_INFO: `/v3/users/3a80899bbb324477ab57749f6b704696` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:56.443806 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce95d76-b94d-4926-b8b5-c886b012854f None admin] RBAC: Authorizing `identity:get_user(user_id=3a80899bbb324477ab57749f6b704696)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:56.445248 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce95d76-b94d-4926-b8b5-c886b012854f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:56.446824 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 596/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:19:56 2022] GET /identity/v3/users/3a80899bbb324477ab57749f6b704696 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:56.452527 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92822888-e8c0-4715-95d9-621d3def5243 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:56.541771 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:56.548488 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92822888-e8c0-4715-95d9-621d3def5243 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:56.582411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4f6c5-e2b3-4efa-ae13-4403d10ea5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:56.583604 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4f6c5-e2b3-4efa-ae13-4403d10ea5b4 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:56.583883 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4f6c5-e2b3-4efa-ae13-4403d10ea5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:56.584154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4f6c5-e2b3-4efa-ae13-4403d10ea5b4 None admin] PATH_INFO: `/v3/projects/6935c5cdbbaa4df09abbb28eba39d724` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:56.585341 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4f6c5-e2b3-4efa-ae13-4403d10ea5b4 None admin] RBAC: Authorizing `identity:get_project(project_id=6935c5cdbbaa4df09abbb28eba39d724)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:56.586785 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4f6c5-e2b3-4efa-ae13-4403d10ea5b4 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:56.588638 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 595/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:19:56 2022] GET /identity/v3/projects/6935c5cdbbaa4df09abbb28eba39d724 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:56.594504 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce95d76-b94d-4926-b8b5-c886b012854f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:56.691306 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce95d76-b94d-4926-b8b5-c886b012854f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:56.723841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7174eba-ad21-4a4b-a4c4-46fab7243b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:56.724511 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7174eba-ad21-4a4b-a4c4-46fab7243b44 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:56.724789 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7174eba-ad21-4a4b-a4c4-46fab7243b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:56.725058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7174eba-ad21-4a4b-a4c4-46fab7243b44 None admin] PATH_INFO: `/v3/projects/6935c5cdbbaa4df09abbb28eba39d724/users/3a80899bbb324477ab57749f6b704696/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:56.726946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7174eba-ad21-4a4b-a4c4-46fab7243b44 None admin] RBAC: Authorizing `identity:create_grant(project_id=6935c5cdbbaa4df09abbb28eba39d724, user_id=3a80899bbb324477ab57749f6b704696, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:56.728372 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7174eba-ad21-4a4b-a4c4-46fab7243b44 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:56.741128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 597/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:19:56 2022] PUT /identity/v3/projects/6935c5cdbbaa4df09abbb28eba39d724/users/3a80899bbb324477ab57749f6b704696/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:19:58.685708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa5d8c-e4ea-4b81-9501-19da91f01dd3 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:58.686250 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa5d8c-e4ea-4b81-9501-19da91f01dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:58.686250 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa5d8c-e4ea-4b81-9501-19da91f01dd3 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:58.687596 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 596/1193] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:19:58.693232 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435c1cd-9fcf-49d8-b10b-1883ec6e6ffd None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:58.693644 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435c1cd-9fcf-49d8-b10b-1883ec6e6ffd None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:58.693755 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435c1cd-9fcf-49d8-b10b-1883ec6e6ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:58.711745 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3435c1cd-9fcf-49d8-b10b-1883ec6e6ffd None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:19:58.713012 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3435c1cd-9fcf-49d8-b10b-1883ec6e6ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:19:58.763333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:58.778346 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 598/1194] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:19:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:19:58.800815 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa5d8c-e4ea-4b81-9501-19da91f01dd3 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:58.899462 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa5d8c-e4ea-4b81-9501-19da91f01dd3 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:58.931318 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e5575-c5bc-4427-bb1c-e4faae872d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:58.931929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e5575-c5bc-4427-bb1c-e4faae872d9f None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:58.932205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e5575-c5bc-4427-bb1c-e4faae872d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:58.932500 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e5575-c5bc-4427-bb1c-e4faae872d9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:58.942229 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e5575-c5bc-4427-bb1c-e4faae872d9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:19:58.947020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-593e5575-c5bc-4427-bb1c-e4faae872d9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:19:58.947020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:19:58.947020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:19:58.947020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:19:58.947020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:19:58.947020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:19:58.947020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:19:58.947020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:19:58.947020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:19:58.947020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:19:58.947020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:19:58.947020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:19:58.947020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:19:58.947020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:19:58.947020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:19:58.947020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:19:58.948194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:19:58.948194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:19:58.948194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:19:58.948194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:19:58.948194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:19:58.948194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:19:58.948194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:19:58.948194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:19:58.948194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:19:58.948194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:19:58.948194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:19:58.948194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:19:58.948194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:19:58.948194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:19:58.948194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:19:58.948194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:19:58.949309 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:19:58.949309 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:19:58.949309 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:19:58.949309 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:19:58.949309 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:19:58.949309 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:19:58.949309 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:19:58.949309 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:19:58.949309 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:19:58.949309 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:19:58.949309 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:19:58.949309 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:19:58.949309 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 597/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:19:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:19:58.954169 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435c1cd-9fcf-49d8-b10b-1883ec6e6ffd None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:59.051631 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435c1cd-9fcf-49d8-b10b-1883ec6e6ffd None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:59.084047 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ab082-9ad2-4e51-b782-386cd9c14f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:59.084751 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ab082-9ad2-4e51-b782-386cd9c14f0a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:59.085032 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ab082-9ad2-4e51-b782-386cd9c14f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:59.085303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ab082-9ad2-4e51-b782-386cd9c14f0a None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:59.086012 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ab082-9ad2-4e51-b782-386cd9c14f0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:59.086312 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ab082-9ad2-4e51-b782-386cd9c14f0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:59.087660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ab082-9ad2-4e51-b782-386cd9c14f0a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:59.095534 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 599/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:19:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:59.101995 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e5575-c5bc-4427-bb1c-e4faae872d9f None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:59.126339 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:59.204413 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e5575-c5bc-4427-bb1c-e4faae872d9f None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:59.237869 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992365a5-1177-4a98-80a5-3bc841cb9fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:59.238485 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992365a5-1177-4a98-80a5-3bc841cb9fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:59.238762 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992365a5-1177-4a98-80a5-3bc841cb9fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:59.239031 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992365a5-1177-4a98-80a5-3bc841cb9fe3 None admin] PATH_INFO: `/v3/users/3a80899bbb324477ab57749f6b704696` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:59.240161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992365a5-1177-4a98-80a5-3bc841cb9fe3 None admin] RBAC: Authorizing `identity:get_user(user_id=3a80899bbb324477ab57749f6b704696)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:59.241883 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992365a5-1177-4a98-80a5-3bc841cb9fe3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:59.243440 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 598/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Sat Aug 6 12:19:59 2022] GET /identity/v3/users/3a80899bbb324477ab57749f6b704696 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:59.249218 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ab082-9ad2-4e51-b782-386cd9c14f0a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:59.297899 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:59.346043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ab082-9ad2-4e51-b782-386cd9c14f0a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:59.377936 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2fd42-10c4-42f2-82d3-3696e1eda206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:59.378527 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2fd42-10c4-42f2-82d3-3696e1eda206 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:59.378808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2fd42-10c4-42f2-82d3-3696e1eda206 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:59.379082 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2fd42-10c4-42f2-82d3-3696e1eda206 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:59.443416 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a2fd42-10c4-42f2-82d3-3696e1eda206 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:59.444872 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a2fd42-10c4-42f2-82d3-3696e1eda206 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:59.490648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:59.539239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 600/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:19:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 06 12:19:59.545088 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992365a5-1177-4a98-80a5-3bc841cb9fe3 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:59.630726 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:19:59.641388 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992365a5-1177-4a98-80a5-3bc841cb9fe3 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:59.675853 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43af76b-3ae0-4e33-8dbf-f0a830e80daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:59.676557 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43af76b-3ae0-4e33-8dbf-f0a830e80daf None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:59.676833 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43af76b-3ae0-4e33-8dbf-f0a830e80daf None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:59.677102 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43af76b-3ae0-4e33-8dbf-f0a830e80daf None admin] PATH_INFO: `/v3/projects/6935c5cdbbaa4df09abbb28eba39d724` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:59.678257 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43af76b-3ae0-4e33-8dbf-f0a830e80daf None admin] RBAC: Authorizing `identity:get_project(project_id=6935c5cdbbaa4df09abbb28eba39d724)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:59.679683 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43af76b-3ae0-4e33-8dbf-f0a830e80daf None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:59.681356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 599/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Sat Aug 6 12:19:59 2022] GET /identity/v3/projects/6935c5cdbbaa4df09abbb28eba39d724 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:19:59.687434 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2fd42-10c4-42f2-82d3-3696e1eda206 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:19:59.783030 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2fd42-10c4-42f2-82d3-3696e1eda206 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:19:59.815439 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b964e75-b6f7-4987-a7ae-683657e9f7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:19:59.816036 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b964e75-b6f7-4987-a7ae-683657e9f7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:19:59.816324 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b964e75-b6f7-4987-a7ae-683657e9f7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:19:59.816606 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b964e75-b6f7-4987-a7ae-683657e9f7b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:19:59.817377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b964e75-b6f7-4987-a7ae-683657e9f7b4 None admin] RBAC: Adding query filter params (role.id=144c8425688b409384e1e8ed8279d88e, scope.project.id=6935c5cdbbaa4df09abbb28eba39d724, user.id=3a80899bbb324477ab57749f6b704696) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:19:59.817685 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b964e75-b6f7-4987-a7ae-683657e9f7b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:19:59.819088 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b964e75-b6f7-4987-a7ae-683657e9f7b4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:19:59.824677 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 601/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:19:59 2022] GET /identity/v3/role_assignments?user.id=3a80899bbb324477ab57749f6b704696&scope.project.id=6935c5cdbbaa4df09abbb28eba39d724&role.id=144c8425688b409384e1e8ed8279d88e => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:20:09.671567 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=21 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:20:09.822089 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=93 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:20:54.258937 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54a8d5-a29a-4651-b66b-26b5c9832259 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:20:54.259490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54a8d5-a29a-4651-b66b-26b5c9832259 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:20:54.259490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54a8d5-a29a-4651-b66b-26b5c9832259 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:20:54.260994 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 600/1201] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:20:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:20:54.272924 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08960369-6504-414b-a4c6-d7f0e50f40a2 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:20:54.275154 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08960369-6504-414b-a4c6-d7f0e50f40a2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:20:54.275438 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08960369-6504-414b-a4c6-d7f0e50f40a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:20:54.294919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-08960369-6504-414b-a4c6-d7f0e50f40a2 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:20:54.296399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-08960369-6504-414b-a4c6-d7f0e50f40a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:20:54.398183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 602/1202] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:20:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:20:54.423478 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54a8d5-a29a-4651-b66b-26b5c9832259 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:20:54.562594 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54a8d5-a29a-4651-b66b-26b5c9832259 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:20:54.611634 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3e788-97eb-4f2f-be8a-0237782a1bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:20:54.612273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3e788-97eb-4f2f-be8a-0237782a1bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:20:54.612581 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3e788-97eb-4f2f-be8a-0237782a1bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:20:54.612855 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3e788-97eb-4f2f-be8a-0237782a1bb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:20:54.624741 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3e788-97eb-4f2f-be8a-0237782a1bb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:20:54.626196 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3e788-97eb-4f2f-be8a-0237782a1bb3 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:20:54.642037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-f7c3e788-97eb-4f2f-be8a-0237782a1bb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:20:54.642037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:20:54.642037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:20:54.642037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:20:54.642037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:20:54.642037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:20:54.642037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:20:54.642037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:20:54.642037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:20:54.642037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:20:54.642037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:20:54.642037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:20:54.642037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:20:54.642037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:20:54.642037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:20:54.643252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:20:54.643252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:20:54.643252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:20:54.643252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:20:54.643252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:20:54.643252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:20:54.643252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:20:54.643252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:20:54.643252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:20:54.643252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:20:54.643252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:20:54.643252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:20:54.643252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:20:54.643252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:20:54.643252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:20:54.643252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:20:54.643252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:20:54.644433 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:20:54.644433 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:20:54.644433 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:20:54.644433 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:20:54.648248 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 601/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:20:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:20:54.649667 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08960369-6504-414b-a4c6-d7f0e50f40a2 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:20:54.774452 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08960369-6504-414b-a4c6-d7f0e50f40a2 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:20:54.812462 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:20:54.819551 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b1d00-c70b-4fce-8c8d-4461c0560ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:20:54.820192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b1d00-c70b-4fce-8c8d-4461c0560ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:20:54.820503 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b1d00-c70b-4fce-8c8d-4461c0560ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:20:54.820776 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b1d00-c70b-4fce-8c8d-4461c0560ff3 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:20:54.821541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b1d00-c70b-4fce-8c8d-4461c0560ff3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:20:54.821840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b1d00-c70b-4fce-8c8d-4461c0560ff3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:20:54.825395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b1d00-c70b-4fce-8c8d-4461c0560ff3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:20:54.839185 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 603/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:20:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:20:54.846030 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3e788-97eb-4f2f-be8a-0237782a1bb3 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:20:54.926286 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:20:54.991432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3e788-97eb-4f2f-be8a-0237782a1bb3 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:20:55.043898 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6bddf-d483-45be-bb8f-298bac95e831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:20:55.044637 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6bddf-d483-45be-bb8f-298bac95e831 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:20:55.044921 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6bddf-d483-45be-bb8f-298bac95e831 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:20:55.045193 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6bddf-d483-45be-bb8f-298bac95e831 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:20:55.045870 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6bddf-d483-45be-bb8f-298bac95e831 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:20:55.047747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6bddf-d483-45be-bb8f-298bac95e831 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:20:55.080147 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 602/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Sat Aug 6 12:20:54 2022] POST /identity/v3/users => generated 251 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:20:57.572642 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a503aa-7362-484c-9b08-c273e942a199 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:20:57.573313 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a503aa-7362-484c-9b08-c273e942a199 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:20:57.573444 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a503aa-7362-484c-9b08-c273e942a199 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:20:57.574719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 604/1206] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:20:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:20:57.580747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe11081-18d1-49fd-80c0-27e08f646c6e None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:20:57.581234 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe11081-18d1-49fd-80c0-27e08f646c6e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:20:57.581356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe11081-18d1-49fd-80c0-27e08f646c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:20:57.605059 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-fbe11081-18d1-49fd-80c0-27e08f646c6e None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:20:57.606461 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fbe11081-18d1-49fd-80c0-27e08f646c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:20:57.695848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 603/1207] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:20:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:20:57.717456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a503aa-7362-484c-9b08-c273e942a199 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:20:57.832712 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a503aa-7362-484c-9b08-c273e942a199 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:20:57.869658 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15da5cb-15ec-4709-ad9c-06f945b1d53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:20:57.870337 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15da5cb-15ec-4709-ad9c-06f945b1d53f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:20:57.870589 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15da5cb-15ec-4709-ad9c-06f945b1d53f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:20:57.870863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15da5cb-15ec-4709-ad9c-06f945b1d53f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:20:57.883339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15da5cb-15ec-4709-ad9c-06f945b1d53f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:20:57.888272 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-a15da5cb-15ec-4709-ad9c-06f945b1d53f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:20:57.888272 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:20:57.888272 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:20:57.888272 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:20:57.888272 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:20:57.888272 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:20:57.888272 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:20:57.888272 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:20:57.888272 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:20:57.888272 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:20:57.888272 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:20:57.888272 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:20:57.888272 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:20:57.888272 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:20:57.888272 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:20:57.888272 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:20:57.889381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:20:57.889381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:20:57.889381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:20:57.889381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:20:57.889381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:20:57.889381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:20:57.889381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:20:57.889381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:20:57.889381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:20:57.889381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:20:57.889381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:20:57.889381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:20:57.889381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:20:57.889381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:20:57.889381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:20:57.889381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:20:57.897952 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:20:57.897952 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:20:57.897952 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:20:57.897952 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:20:57.897952 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:20:57.897952 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:20:57.897952 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:20:57.897952 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:20:57.897952 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:20:57.897952 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:20:57.897952 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:20:57.897952 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:20:57.897952 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 605/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Aug 6 12:20:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:20:57.898858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe11081-18d1-49fd-80c0-27e08f646c6e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:20:57.951136 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:20:58.032827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe11081-18d1-49fd-80c0-27e08f646c6e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:20:58.071399 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78071fb5-e30e-4f98-8894-9051cabdd3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:20:58.072098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78071fb5-e30e-4f98-8894-9051cabdd3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:20:58.072327 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78071fb5-e30e-4f98-8894-9051cabdd3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:20:58.072617 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78071fb5-e30e-4f98-8894-9051cabdd3e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:20:58.073371 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78071fb5-e30e-4f98-8894-9051cabdd3e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:20:58.073649 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78071fb5-e30e-4f98-8894-9051cabdd3e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:20:58.075020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78071fb5-e30e-4f98-8894-9051cabdd3e9 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:20:58.083713 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 604/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Aug 6 12:20:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:20:58.089807 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15da5cb-15ec-4709-ad9c-06f945b1d53f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:20:58.151399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:20:58.204759 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15da5cb-15ec-4709-ad9c-06f945b1d53f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:20:58.243084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9479e4-afe4-417d-b4b5-5452dba66817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:20:58.243746 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9479e4-afe4-417d-b4b5-5452dba66817 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:20:58.244031 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9479e4-afe4-417d-b4b5-5452dba66817 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:20:58.244336 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9479e4-afe4-417d-b4b5-5452dba66817 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:20:58.256895 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9479e4-afe4-417d-b4b5-5452dba66817 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:20:58.258635 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9479e4-afe4-417d-b4b5-5452dba66817 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:20:58.289880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-8c9479e4-afe4-417d-b4b5-5452dba66817 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:20:58.289880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:20:58.289880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:20:58.289880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:20:58.289880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:20:58.289880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:20:58.289880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:20:58.289880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:20:58.289880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:20:58.289880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:20:58.289880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:20:58.289880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:20:58.289880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:20:58.289880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:20:58.289880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:20:58.291400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:20:58.291400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:20:58.291400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:20:58.291400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:20:58.291400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:20:58.291400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:20:58.291400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:20:58.291400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:20:58.291400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:20:58.291400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:20:58.291400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:20:58.291400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:20:58.291400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:20:58.291400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:20:58.291400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:20:58.291400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:20:58.291400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:20:58.292515 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:20:58.292515 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:20:58.292515 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:20:58.292515 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:20:58.293682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 606/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:20:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:20:58.301304 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78071fb5-e30e-4f98-8894-9051cabdd3e9 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:20:58.416769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:20:58.419856 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78071fb5-e30e-4f98-8894-9051cabdd3e9 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:20:58.464032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9233b5-126a-43d1-b065-c236fa25c752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:20:58.464704 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9233b5-126a-43d1-b065-c236fa25c752 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:20:58.464988 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9233b5-126a-43d1-b065-c236fa25c752 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:20:58.465281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9233b5-126a-43d1-b065-c236fa25c752 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:20:58.465976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9233b5-126a-43d1-b065-c236fa25c752 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:20:58.466282 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9233b5-126a-43d1-b065-c236fa25c752 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:20:58.467649 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9233b5-126a-43d1-b065-c236fa25c752 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:20:58.481601 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 605/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:20:58.489175 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9479e4-afe4-417d-b4b5-5452dba66817 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:20:58.619328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9479e4-afe4-417d-b4b5-5452dba66817 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:20:58.621604 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:20:58.663616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad0964-d39c-4a70-9ae1-5284f3d4e9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:20:58.665082 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad0964-d39c-4a70-9ae1-5284f3d4e9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:20:58.665082 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad0964-d39c-4a70-9ae1-5284f3d4e9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:20:58.665301 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad0964-d39c-4a70-9ae1-5284f3d4e9d2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:20:58.674815 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad0964-d39c-4a70-9ae1-5284f3d4e9d2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:20:58.676942 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad0964-d39c-4a70-9ae1-5284f3d4e9d2 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:20:58.690046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-99ad0964-d39c-4a70-9ae1-5284f3d4e9d2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 12:20:58.690046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:20:58.690046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:20:58.690046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:20:58.690046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:20:58.690046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:20:58.690046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:20:58.690046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:20:58.690046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:20:58.690046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:20:58.690046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:20:58.690046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:20:58.690046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:20:58.690046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:20:58.690046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:20:58.690046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:20:58.691257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:20:58.691257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:20:58.691257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:20:58.691257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:20:58.691257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:20:58.691257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:20:58.691257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:20:58.691257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:20:58.691257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:20:58.691257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:20:58.691257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:20:58.691257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:20:58.691257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:20:58.691257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:20:58.691257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:20:58.691257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:20:58.691257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:20:58.691257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:20:58.692599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:20:58.692599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:20:58.692599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:20:58.692599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:20:58.692599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:20:58.692599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:20:58.692599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:20:58.692599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:20:58.692599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 12:20:58.692599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:20:58.692599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 607/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:20:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:20:58.697441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9233b5-126a-43d1-b065-c236fa25c752 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:20:58.818142 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9233b5-126a-43d1-b065-c236fa25c752 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:20:58.862807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a475590-b9ff-4ce8-a608-f96db98bee85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:20:58.863483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a475590-b9ff-4ce8-a608-f96db98bee85 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:20:58.863944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a475590-b9ff-4ce8-a608-f96db98bee85 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:20:58.864270 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a475590-b9ff-4ce8-a608-f96db98bee85 None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:20:58.865394 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a475590-b9ff-4ce8-a608-f96db98bee85 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:20:58.865745 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a475590-b9ff-4ce8-a608-f96db98bee85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:20:58.867338 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a475590-b9ff-4ce8-a608-f96db98bee85 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:20:58.894038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 606/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Sat Aug 6 12:20:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:20:58.895560 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad0964-d39c-4a70-9ae1-5284f3d4e9d2 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:20:59.046224 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad0964-d39c-4a70-9ae1-5284f3d4e9d2 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:20:59.090476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d9b37-5339-4d41-9da8-c99bbda3e5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:20:59.091126 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d9b37-5339-4d41-9da8-c99bbda3e5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:20:59.091390 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d9b37-5339-4d41-9da8-c99bbda3e5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:20:59.091663 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d9b37-5339-4d41-9da8-c99bbda3e5e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:20:59.126029 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:20:59.172602 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d9b37-5339-4d41-9da8-c99bbda3e5e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:20:59.174202 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d9b37-5339-4d41-9da8-c99bbda3e5e5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:20:59.287705 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 608/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:20:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 12:20:59.293682 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a475590-b9ff-4ce8-a608-f96db98bee85 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:20:59.315258 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:20:59.416323 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a475590-b9ff-4ce8-a608-f96db98bee85 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:20:59.453515 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cacf81-c96d-41d6-9d29-aadd344552a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:20:59.454201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cacf81-c96d-41d6-9d29-aadd344552a7 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:20:59.455093 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cacf81-c96d-41d6-9d29-aadd344552a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:20:59.455366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cacf81-c96d-41d6-9d29-aadd344552a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:20:59.467211 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cacf81-c96d-41d6-9d29-aadd344552a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:20:59.468805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cacf81-c96d-41d6-9d29-aadd344552a7 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:20:59.486411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-56cacf81-c96d-41d6-9d29-aadd344552a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:20:59.486411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:20:59.486411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:20:59.486411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:20:59.486411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:20:59.486411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:20:59.486411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:20:59.486411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:20:59.486411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:20:59.486411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:20:59.486411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:20:59.486411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:20:59.486411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:20:59.486411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:20:59.486411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:20:59.487548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:20:59.487548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:20:59.487548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:20:59.487548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:20:59.487548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:20:59.487548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:20:59.487548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:20:59.487548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:20:59.487548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:20:59.487548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:20:59.487548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:20:59.487548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:20:59.487548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:20:59.487548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:20:59.487548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:20:59.487548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:20:59.487548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:20:59.490011 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:20:59.490011 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:20:59.490011 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:20:59.490011 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:20:59.490011 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 607/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:20:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:20:59.493756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d9b37-5339-4d41-9da8-c99bbda3e5e5 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:20:59.559327 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:20:59.604207 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d9b37-5339-4d41-9da8-c99bbda3e5e5 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:20:59.643471 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c6c71-f5a4-4bd9-8e39-f945c02a50a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:20:59.644238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c6c71-f5a4-4bd9-8e39-f945c02a50a3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:20:59.644370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c6c71-f5a4-4bd9-8e39-f945c02a50a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:20:59.644635 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c6c71-f5a4-4bd9-8e39-f945c02a50a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:20:59.645403 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c6c71-f5a4-4bd9-8e39-f945c02a50a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:20:59.645705 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c6c71-f5a4-4bd9-8e39-f945c02a50a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:20:59.647132 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c6c71-f5a4-4bd9-8e39-f945c02a50a3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:20:59.664794 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 609/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:20:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:20:59.666048 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cacf81-c96d-41d6-9d29-aadd344552a7 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:20:59.762329 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:20:59.799752 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cacf81-c96d-41d6-9d29-aadd344552a7 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:20:59.845735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac6a74-b343-450e-abc1-f01c8b368338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:20:59.846290 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac6a74-b343-450e-abc1-f01c8b368338 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:20:59.846570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac6a74-b343-450e-abc1-f01c8b368338 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:20:59.846845 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac6a74-b343-450e-abc1-f01c8b368338 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:20:59.854765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac6a74-b343-450e-abc1-f01c8b368338 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:20:59.856538 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac6a74-b343-450e-abc1-f01c8b368338 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:20:59.870357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-bcac6a74-b343-450e-abc1-f01c8b368338 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:20:59.870357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:20:59.870357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:20:59.870357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:20:59.870357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:20:59.870357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:20:59.870357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:20:59.870357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:20:59.870357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:20:59.870357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:20:59.870357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:20:59.870357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:20:59.870357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:20:59.870357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:20:59.870357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:20:59.871366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:20:59.871366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:20:59.871366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:20:59.871366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:20:59.871366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:20:59.871366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:20:59.871366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:20:59.871366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:20:59.871366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:20:59.871366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:20:59.871366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:20:59.871366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:20:59.871366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:20:59.871366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:20:59.871366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:20:59.871366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:20:59.871366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:20:59.872284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:20:59.872284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:20:59.872284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:20:59.872284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:20:59.872284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:20:59.872284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:20:59.872284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:20:59.872284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:20:59.872284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 608/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:20:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:20:59.878685 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c6c71-f5a4-4bd9-8e39-f945c02a50a3 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:00.010495 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c6c71-f5a4-4bd9-8e39-f945c02a50a3 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:00.049839 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:00.067115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33bb8d0-6d3e-4b95-a5a7-9347ee2736a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:00.067757 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33bb8d0-6d3e-4b95-a5a7-9347ee2736a9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:00.068022 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33bb8d0-6d3e-4b95-a5a7-9347ee2736a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:00.068295 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33bb8d0-6d3e-4b95-a5a7-9347ee2736a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:00.069075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33bb8d0-6d3e-4b95-a5a7-9347ee2736a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:00.069383 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33bb8d0-6d3e-4b95-a5a7-9347ee2736a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:00.071101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33bb8d0-6d3e-4b95-a5a7-9347ee2736a9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:00.084706 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 610/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:20:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:00.093219 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac6a74-b343-450e-abc1-f01c8b368338 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:00.228968 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac6a74-b343-450e-abc1-f01c8b368338 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:00.267562 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:00.274039 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc9efe-a3bf-4027-954e-9d74c6211f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:00.275031 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc9efe-a3bf-4027-954e-9d74c6211f20 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:00.276158 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc9efe-a3bf-4027-954e-9d74c6211f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:00.276921 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc9efe-a3bf-4027-954e-9d74c6211f20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:00.278311 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cc9efe-a3bf-4027-954e-9d74c6211f20 None admin] RBAC: Adding query filter params (role.id=785cca921bad4984a86e45ce6c5601c2, scope.project.id=13fba33c46cf40bbb15a08c690b4c98c, user.id=e2244655ba304b9c980b5e68772eefc0) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:00.278726 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cc9efe-a3bf-4027-954e-9d74c6211f20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:00.280961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cc9efe-a3bf-4027-954e-9d74c6211f20 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:00.289534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 609/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:21:00 2022] GET /identity/v3/role_assignments?user.id=e2244655ba304b9c980b5e68772eefc0&scope.project.id=13fba33c46cf40bbb15a08c690b4c98c&role.id=785cca921bad4984a86e45ce6c5601c2 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:02.389325 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90016f-180b-462d-88b1-176b6221160f None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:02.389899 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90016f-180b-462d-88b1-176b6221160f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:02.390005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90016f-180b-462d-88b1-176b6221160f None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:02.392163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 611/1220] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:21:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:21:02.397512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc9c1c-f32b-4030-99f5-a71d1f89a6e5 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:02.398013 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc9c1c-f32b-4030-99f5-a71d1f89a6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:02.398155 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc9c1c-f32b-4030-99f5-a71d1f89a6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:02.417794 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-b3cc9c1c-f32b-4030-99f5-a71d1f89a6e5 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:21:02.419232 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b3cc9c1c-f32b-4030-99f5-a71d1f89a6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:21:02.491803 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 610/1221] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:21:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 06 12:21:02.518100 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90016f-180b-462d-88b1-176b6221160f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:02.624225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90016f-180b-462d-88b1-176b6221160f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:02.666879 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941aeb97-119f-4050-93d0-282492d21979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:02.667923 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941aeb97-119f-4050-93d0-282492d21979 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:02.668380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941aeb97-119f-4050-93d0-282492d21979 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:02.669594 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941aeb97-119f-4050-93d0-282492d21979 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:02.683739 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941aeb97-119f-4050-93d0-282492d21979 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:21:02.691007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-941aeb97-119f-4050-93d0-282492d21979 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:21:02.691007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:02.691007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:02.691007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:02.691007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:02.691007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:02.691007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:02.691007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:02.691007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:02.691007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:02.691007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:02.691007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:02.691007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:21:02.691007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:21:02.691007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:21:02.691007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:21:02.692093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:21:02.692093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:21:02.692093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:21:02.692093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:21:02.692093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:02.692093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:02.692093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:02.692093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:02.692093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:02.692093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:02.692093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:02.692093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:02.692093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:02.692093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:02.692093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:02.692093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:02.693143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:02.693143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:02.693143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:02.693143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:02.693143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:21:02.693143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:21:02.693143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:21:02.693143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:21:02.693143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:21:02.693143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:21:02.693143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:21:02.693143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:21:02.693143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 612/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Aug 6 12:21:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:21:02.698934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc9c1c-f32b-4030-99f5-a71d1f89a6e5 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:02.807319 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc9c1c-f32b-4030-99f5-a71d1f89a6e5 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:02.831021 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:02.843705 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8582f0-7a01-4f1f-bf9b-cfc6d06c70fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:02.844345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8582f0-7a01-4f1f-bf9b-cfc6d06c70fc None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:02.844654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8582f0-7a01-4f1f-bf9b-cfc6d06c70fc None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:02.844928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8582f0-7a01-4f1f-bf9b-cfc6d06c70fc None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:02.845644 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8582f0-7a01-4f1f-bf9b-cfc6d06c70fc None admin] RBAC: Adding query filter params (name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:02.845946 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8582f0-7a01-4f1f-bf9b-cfc6d06c70fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:02.847302 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8582f0-7a01-4f1f-bf9b-cfc6d06c70fc None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:02.856570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 611/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Aug 6 12:21:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:02.862898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941aeb97-119f-4050-93d0-282492d21979 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:02.909719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:02.968261 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941aeb97-119f-4050-93d0-282492d21979 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:03.008776 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5cc890-a154-4743-9835-79b3f64ac1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:03.009444 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cc890-a154-4743-9835-79b3f64ac1ca None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:03.009734 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cc890-a154-4743-9835-79b3f64ac1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:03.010025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cc890-a154-4743-9835-79b3f64ac1ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:03.024989 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5cc890-a154-4743-9835-79b3f64ac1ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:03.026425 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5cc890-a154-4743-9835-79b3f64ac1ca None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:03.042541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-ca5cc890-a154-4743-9835-79b3f64ac1ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:21:03.042541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:03.042541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:03.042541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:03.042541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:03.042541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:03.042541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:03.042541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:03.042541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:03.042541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:03.042541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:03.042541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:03.042541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:21:03.042541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:21:03.042541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:21:03.043536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:21:03.043536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:03.043536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:03.043536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:03.043536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:03.043536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:03.043536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:03.043536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:03.043536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:03.043536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:03.043536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:03.043536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:03.043536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:03.043536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:03.043536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:03.043536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:03.043536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:03.049755 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:21:03.049755 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:21:03.049755 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:21:03.049755 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:21:03.049755 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 613/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:21:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:21:03.050047 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8582f0-7a01-4f1f-bf9b-cfc6d06c70fc None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:03.159130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8582f0-7a01-4f1f-bf9b-cfc6d06c70fc None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:03.195203 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e93fc-3689-4dd2-aba7-bb513ad5e08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:03.195830 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e93fc-3689-4dd2-aba7-bb513ad5e08b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:03.196108 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e93fc-3689-4dd2-aba7-bb513ad5e08b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:03.196423 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e93fc-3689-4dd2-aba7-bb513ad5e08b None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:03.197126 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e93fc-3689-4dd2-aba7-bb513ad5e08b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:03.197428 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e93fc-3689-4dd2-aba7-bb513ad5e08b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:03.198867 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e93fc-3689-4dd2-aba7-bb513ad5e08b None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:03.210797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 612/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:03.216845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5cc890-a154-4743-9835-79b3f64ac1ca None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:03.323196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:03.330796 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5cc890-a154-4743-9835-79b3f64ac1ca None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:03.375079 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58e30a7-58d6-4c07-bba5-403997aa9a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:03.375865 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e30a7-58d6-4c07-bba5-403997aa9a27 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:03.376142 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e30a7-58d6-4c07-bba5-403997aa9a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:03.376491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e30a7-58d6-4c07-bba5-403997aa9a27 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:03.385101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58e30a7-58d6-4c07-bba5-403997aa9a27 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:03.386654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58e30a7-58d6-4c07-bba5-403997aa9a27 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:03.399167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-a58e30a7-58d6-4c07-bba5-403997aa9a27 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 12:21:03.399167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:03.399167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:03.399167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:03.399167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:03.399167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:03.399167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:03.399167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:03.399167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:03.399167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:03.399167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:03.399167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:03.399167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:21:03.399167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:21:03.399167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:21:03.399167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:21:03.401246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:03.401246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:03.401246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:21:03.401246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:21:03.401246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:21:03.401246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:21:03.401246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:03.401246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:03.401246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:03.401246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:03.401246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:03.401246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:03.401246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:03.401246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:03.401246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:03.401246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:03.401246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:03.401246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:03.402523 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:03.402523 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:03.402523 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:21:03.402523 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:21:03.402523 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:21:03.402523 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:21:03.402523 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:21:03.402523 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:21:03.402523 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 12:21:03.402523 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:21:03.402523 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 614/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:21:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:21:03.407851 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e93fc-3689-4dd2-aba7-bb513ad5e08b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:03.463970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:03.524925 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e93fc-3689-4dd2-aba7-bb513ad5e08b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:03.565981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528151af-4392-4bcb-8147-10996ed557aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:03.566738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528151af-4392-4bcb-8147-10996ed557aa None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:03.567050 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528151af-4392-4bcb-8147-10996ed557aa None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:03.567533 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528151af-4392-4bcb-8147-10996ed557aa None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:03.568563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528151af-4392-4bcb-8147-10996ed557aa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:03.568869 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528151af-4392-4bcb-8147-10996ed557aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:03.570620 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528151af-4392-4bcb-8147-10996ed557aa None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:03.596203 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 613/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Sat Aug 6 12:21:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:03.597127 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58e30a7-58d6-4c07-bba5-403997aa9a27 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:03.722927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58e30a7-58d6-4c07-bba5-403997aa9a27 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:03.758770 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b905ae-c0fb-4360-b8f8-50e913e0f671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:03.759469 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b905ae-c0fb-4360-b8f8-50e913e0f671 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:03.759769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b905ae-c0fb-4360-b8f8-50e913e0f671 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:03.760042 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b905ae-c0fb-4360-b8f8-50e913e0f671 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:03.773861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b905ae-c0fb-4360-b8f8-50e913e0f671 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:03.775358 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b905ae-c0fb-4360-b8f8-50e913e0f671 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:03.790058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-c3b905ae-c0fb-4360-b8f8-50e913e0f671 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:21:03.790058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:03.790058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:03.790058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:03.790058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:03.790058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:03.790058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:03.790058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:03.790058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:03.790058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:03.790058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:03.790058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:03.790058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:21:03.790058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:21:03.790058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:21:03.791199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:21:03.791199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:03.791199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:03.791199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:03.791199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:03.791199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:03.791199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:03.791199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:03.791199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:03.791199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:03.791199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:03.791199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:03.791199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:03.791199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:03.791199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:03.791199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:03.791199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:03.792140 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:21:03.792140 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:21:03.792140 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:21:03.792140 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:21:03.792140 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 615/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:21:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:21:03.796998 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528151af-4392-4bcb-8147-10996ed557aa None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:03.909540 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528151af-4392-4bcb-8147-10996ed557aa None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:03.911727 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:03.949872 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c5751-a67d-4d99-8035-b8ced8f21e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:03.950491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c5751-a67d-4d99-8035-b8ced8f21e55 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:03.950796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c5751-a67d-4d99-8035-b8ced8f21e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:03.951039 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c5751-a67d-4d99-8035-b8ced8f21e55 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:03.951723 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c5751-a67d-4d99-8035-b8ced8f21e55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:03.952022 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c5751-a67d-4d99-8035-b8ced8f21e55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:03.953428 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c5751-a67d-4d99-8035-b8ced8f21e55 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:03.965496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 614/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:03.972172 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b905ae-c0fb-4360-b8f8-50e913e0f671 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:03.981675 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:04.093352 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b905ae-c0fb-4360-b8f8-50e913e0f671 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:04.131703 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67218e6-f011-49a0-818e-bb332fe10308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:04.132517 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67218e6-f011-49a0-818e-bb332fe10308 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:04.132850 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67218e6-f011-49a0-818e-bb332fe10308 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:04.133178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67218e6-f011-49a0-818e-bb332fe10308 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:04.141981 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67218e6-f011-49a0-818e-bb332fe10308 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:04.143855 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67218e6-f011-49a0-818e-bb332fe10308 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:04.158927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-a67218e6-f011-49a0-818e-bb332fe10308 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:21:04.158927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:04.158927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:04.158927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:04.158927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:04.158927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:04.158927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:04.158927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:04.158927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:04.158927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:04.158927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:04.158927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:04.158927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:21:04.158927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:21:04.158927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:21:04.162528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:21:04.162528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:04.162528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:04.162528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:04.162528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:04.162528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:04.162528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:04.162528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:04.162528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:04.162528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:04.162528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:04.162528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:04.162528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:04.162528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:04.162528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:04.162528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:04.162528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:04.165644 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:21:04.165644 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:21:04.165644 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:21:04.165644 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:21:04.165644 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:21:04.165644 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:21:04.165644 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:21:04.165644 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:21:04.165644 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 616/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:21:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:21:04.168867 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c5751-a67d-4d99-8035-b8ced8f21e55 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:04.288929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c5751-a67d-4d99-8035-b8ced8f21e55 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:04.326115 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99ce05-211f-47b0-9fe1-16d708a8ab69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:04.326729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99ce05-211f-47b0-9fe1-16d708a8ab69 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:04.327009 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99ce05-211f-47b0-9fe1-16d708a8ab69 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:04.327305 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99ce05-211f-47b0-9fe1-16d708a8ab69 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:04.328203 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99ce05-211f-47b0-9fe1-16d708a8ab69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:04.328554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99ce05-211f-47b0-9fe1-16d708a8ab69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:04.330309 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99ce05-211f-47b0-9fe1-16d708a8ab69 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:04.348328 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 615/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:21:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:04.352852 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67218e6-f011-49a0-818e-bb332fe10308 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:04.431679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:04.471057 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67218e6-f011-49a0-818e-bb332fe10308 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:04.505614 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f1c68-d02f-4872-8662-1c8fc6071c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:04.506371 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f1c68-d02f-4872-8662-1c8fc6071c82 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:04.506669 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f1c68-d02f-4872-8662-1c8fc6071c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:04.506950 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f1c68-d02f-4872-8662-1c8fc6071c82 None admin] PATH_INFO: `/v3/projects/13fba33c46cf40bbb15a08c690b4c98c/users/e2244655ba304b9c980b5e68772eefc0/roles/785cca921bad4984a86e45ce6c5601c2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:04.523691 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f1c68-d02f-4872-8662-1c8fc6071c82 None admin] RBAC: Authorizing `identity:create_grant(project_id=13fba33c46cf40bbb15a08c690b4c98c, user_id=e2244655ba304b9c980b5e68772eefc0, role_id=785cca921bad4984a86e45ce6c5601c2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:04.525157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f1c68-d02f-4872-8662-1c8fc6071c82 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:04.540275 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 617/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:21:04 2022] PUT /identity/v3/projects/13fba33c46cf40bbb15a08c690b4c98c/users/e2244655ba304b9c980b5e68772eefc0/roles/785cca921bad4984a86e45ce6c5601c2 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:21:06.592488 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b8337-6e85-4968-a76e-5584952c0a75 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:06.593052 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b8337-6e85-4968-a76e-5584952c0a75 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:06.593052 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b8337-6e85-4968-a76e-5584952c0a75 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:06.594397 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 616/1233] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:21:06.600539 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de24a7-695f-48fa-b504-3b75aabb3e46 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:06.600982 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de24a7-695f-48fa-b504-3b75aabb3e46 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:06.601102 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de24a7-695f-48fa-b504-3b75aabb3e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:06.632467 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-14de24a7-695f-48fa-b504-3b75aabb3e46 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:21:06.633675 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-14de24a7-695f-48fa-b504-3b75aabb3e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:21:06.716592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 618/1234] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:21:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:21:06.742343 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b8337-6e85-4968-a76e-5584952c0a75 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:06.768634 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:06.854381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b8337-6e85-4968-a76e-5584952c0a75 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:06.898358 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298040f-8d74-4d56-b4a0-a10afe3b92dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:06.899018 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298040f-8d74-4d56-b4a0-a10afe3b92dd None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:06.899295 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298040f-8d74-4d56-b4a0-a10afe3b92dd None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:06.899565 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298040f-8d74-4d56-b4a0-a10afe3b92dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:06.910702 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d298040f-8d74-4d56-b4a0-a10afe3b92dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:21:06.915576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-d298040f-8d74-4d56-b4a0-a10afe3b92dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:21:06.915576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:06.915576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:06.915576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:06.915576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:06.915576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:06.915576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:06.915576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:06.915576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:06.915576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:06.915576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:06.915576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:06.915576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:21:06.915576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:21:06.915576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:21:06.915576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:21:06.916802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:21:06.916802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:21:06.916802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:21:06.916802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:21:06.916802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:06.916802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:06.916802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:06.916802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:06.916802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:06.916802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:06.916802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:06.916802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:06.916802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:06.916802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:06.916802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:06.916802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:06.917989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:06.917989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:06.917989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:06.917989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:06.917989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:21:06.917989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:21:06.917989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:21:06.917989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:21:06.917989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:21:06.917989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:21:06.917989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 06 12:21:06.917989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:21:06.917989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 617/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Sat Aug 6 12:21:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:21:06.923446 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de24a7-695f-48fa-b504-3b75aabb3e46 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:06.973161 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:07.033931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de24a7-695f-48fa-b504-3b75aabb3e46 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:07.069561 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801be94-4b7a-4477-b0ce-1a3d0acbc012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:07.070230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801be94-4b7a-4477-b0ce-1a3d0acbc012 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:07.070591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801be94-4b7a-4477-b0ce-1a3d0acbc012 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:07.070882 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801be94-4b7a-4477-b0ce-1a3d0acbc012 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:07.071725 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b801be94-4b7a-4477-b0ce-1a3d0acbc012 None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:07.072035 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b801be94-4b7a-4477-b0ce-1a3d0acbc012 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:07.073484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b801be94-4b7a-4477-b0ce-1a3d0acbc012 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:07.082289 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 619/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Sat Aug 6 12:21:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:21:07.088019 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298040f-8d74-4d56-b4a0-a10afe3b92dd None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:07.186127 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:07.205652 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298040f-8d74-4d56-b4a0-a10afe3b92dd None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:07.244735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc02b5-6bbb-4399-b9df-f3a4c162fb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:07.245396 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc02b5-6bbb-4399-b9df-f3a4c162fb90 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:07.245687 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc02b5-6bbb-4399-b9df-f3a4c162fb90 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:07.245959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc02b5-6bbb-4399-b9df-f3a4c162fb90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:07.256983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc02b5-6bbb-4399-b9df-f3a4c162fb90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:07.258525 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc02b5-6bbb-4399-b9df-f3a4c162fb90 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:07.274027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-93dc02b5-6bbb-4399-b9df-f3a4c162fb90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:21:07.274027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:07.274027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:07.274027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:07.274027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:07.274027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:07.274027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:07.274027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:07.274027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:07.274027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:07.274027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:07.274027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:07.274027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:21:07.274027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:21:07.274027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:21:07.274976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:21:07.274976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:07.274976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:07.274976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:07.274976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:07.274976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:07.274976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:07.274976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:07.274976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:07.274976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:07.274976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:07.274976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:07.274976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:07.274976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:07.274976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:07.274976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:07.274976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:07.280196 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:21:07.280196 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:21:07.280196 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:21:07.280196 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:21:07.280196 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 618/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:21:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:21:07.281052 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801be94-4b7a-4477-b0ce-1a3d0acbc012 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:07.394654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801be94-4b7a-4477-b0ce-1a3d0acbc012 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:07.405755 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:07.432639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c334167-05f8-4384-9a6a-cc16534b1455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:07.433249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c334167-05f8-4384-9a6a-cc16534b1455 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:07.433527 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c334167-05f8-4384-9a6a-cc16534b1455 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:07.433802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c334167-05f8-4384-9a6a-cc16534b1455 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:07.434485 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c334167-05f8-4384-9a6a-cc16534b1455 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:07.434775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c334167-05f8-4384-9a6a-cc16534b1455 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:07.436816 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c334167-05f8-4384-9a6a-cc16534b1455 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:07.449996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 620/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:21:07.456069 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc02b5-6bbb-4399-b9df-f3a4c162fb90 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:07.565194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc02b5-6bbb-4399-b9df-f3a4c162fb90 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:07.600587 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b1788-1372-4697-99ce-accf03a7701c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:07.601243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b1788-1372-4697-99ce-accf03a7701c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:07.601481 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b1788-1372-4697-99ce-accf03a7701c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:07.601744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b1788-1372-4697-99ce-accf03a7701c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:07.604597 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:07.609905 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b1788-1372-4697-99ce-accf03a7701c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:07.611655 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b1788-1372-4697-99ce-accf03a7701c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:07.626273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-f67b1788-1372-4697-99ce-accf03a7701c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 12:21:07.626273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:07.626273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:07.626273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:07.626273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:07.626273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:07.626273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:07.626273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:07.626273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:07.626273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:07.626273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:07.626273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:07.626273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:21:07.626273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:21:07.626273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:21:07.626273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:21:07.627254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:07.627254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:07.627254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:21:07.627254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:21:07.627254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:21:07.627254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:21:07.627254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:07.627254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:07.627254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:07.627254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:07.627254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:07.627254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:07.627254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:07.627254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:07.627254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:07.627254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:07.627254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:07.627254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:07.628214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:07.628214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:07.628214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:21:07.628214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:21:07.628214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:21:07.628214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:21:07.628214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:21:07.628214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:21:07.628214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 12:21:07.628214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:21:07.628214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 619/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:21:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:21:07.633467 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c334167-05f8-4384-9a6a-cc16534b1455 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:07.760233 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c334167-05f8-4384-9a6a-cc16534b1455 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:07.800241 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957fd94-eab2-4081-bbf6-a0769dc7f660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:07.800889 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957fd94-eab2-4081-bbf6-a0769dc7f660 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:07.801244 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957fd94-eab2-4081-bbf6-a0769dc7f660 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:07.801593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957fd94-eab2-4081-bbf6-a0769dc7f660 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:07.802700 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957fd94-eab2-4081-bbf6-a0769dc7f660 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:07.803031 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957fd94-eab2-4081-bbf6-a0769dc7f660 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:07.804592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957fd94-eab2-4081-bbf6-a0769dc7f660 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:07.824067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 621/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Sat Aug 6 12:21:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:21:07.830196 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b1788-1372-4697-99ce-accf03a7701c None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:07.938329 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b1788-1372-4697-99ce-accf03a7701c None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:07.975938 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95565b-6960-4158-bddb-3973628c0532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:07.976598 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95565b-6960-4158-bddb-3973628c0532 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:07.976877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95565b-6960-4158-bddb-3973628c0532 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:07.977146 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95565b-6960-4158-bddb-3973628c0532 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:08.048289 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:08.055836 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95565b-6960-4158-bddb-3973628c0532 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:08.057316 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95565b-6960-4158-bddb-3973628c0532 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:08.166970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 620/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:21:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 12:21:08.174769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957fd94-eab2-4081-bbf6-a0769dc7f660 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:08.206919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:08.288107 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957fd94-eab2-4081-bbf6-a0769dc7f660 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:08.324558 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf018a-9561-4a45-b33a-ecc5e1228f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:08.324999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf018a-9561-4a45-b33a-ecc5e1228f36 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:08.325323 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf018a-9561-4a45-b33a-ecc5e1228f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:08.325600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf018a-9561-4a45-b33a-ecc5e1228f36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:08.336921 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf018a-9561-4a45-b33a-ecc5e1228f36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:08.338381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf018a-9561-4a45-b33a-ecc5e1228f36 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:08.354775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-1dbf018a-9561-4a45-b33a-ecc5e1228f36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:21:08.354775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:08.354775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:08.354775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:08.354775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:08.354775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:08.354775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:08.354775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:08.354775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:08.354775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:08.354775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:08.354775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:08.354775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:21:08.354775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:21:08.354775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:21:08.355832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:21:08.355832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:08.355832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:08.355832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:08.355832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:08.355832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:08.355832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:08.355832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:08.355832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:08.355832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:08.355832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:08.355832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:08.355832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:08.355832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:08.355832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:08.355832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:08.355832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:08.357100 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:21:08.357100 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:21:08.357100 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:21:08.357100 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:21:08.357100 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 622/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:21:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:21:08.362567 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95565b-6960-4158-bddb-3973628c0532 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:08.458867 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:08.466125 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95565b-6960-4158-bddb-3973628c0532 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:08.504739 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4e1d5-0197-48e2-a07f-5110f93a0f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:08.505353 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4e1d5-0197-48e2-a07f-5110f93a0f61 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:08.505634 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4e1d5-0197-48e2-a07f-5110f93a0f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:08.505910 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4e1d5-0197-48e2-a07f-5110f93a0f61 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:08.506610 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4e1d5-0197-48e2-a07f-5110f93a0f61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:08.506902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4e1d5-0197-48e2-a07f-5110f93a0f61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:08.509498 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4e1d5-0197-48e2-a07f-5110f93a0f61 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:08.523067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 621/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:08.528985 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf018a-9561-4a45-b33a-ecc5e1228f36 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:08.628692 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:08.654614 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf018a-9561-4a45-b33a-ecc5e1228f36 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:08.694455 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850003b6-4e0b-4720-a777-68620b33376b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:08.695337 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850003b6-4e0b-4720-a777-68620b33376b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:08.695745 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850003b6-4e0b-4720-a777-68620b33376b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:08.696123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850003b6-4e0b-4720-a777-68620b33376b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:08.706516 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850003b6-4e0b-4720-a777-68620b33376b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:08.708510 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850003b6-4e0b-4720-a777-68620b33376b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:08.723817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-850003b6-4e0b-4720-a777-68620b33376b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:21:08.723817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:08.723817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:08.723817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:08.723817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:08.723817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:08.723817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:08.723817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:08.723817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:08.723817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:08.723817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:08.723817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:08.723817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:21:08.723817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:21:08.723817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:21:08.725143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:21:08.725143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:08.725143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:08.725143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:08.725143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:08.725143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:08.725143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:08.725143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:08.725143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:08.725143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:08.725143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:08.725143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:08.725143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:08.725143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:08.725143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:08.725143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:08.725143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:08.726764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:21:08.726764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:21:08.726764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:21:08.726764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:21:08.726764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:21:08.726764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:21:08.726764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:21:08.726764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:21:08.729377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 623/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:21:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:21:08.735407 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4e1d5-0197-48e2-a07f-5110f93a0f61 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:08.854166 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4e1d5-0197-48e2-a07f-5110f93a0f61 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:08.889680 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac39f9-2072-41f8-8e0a-3d9fa59e476c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:08.890220 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac39f9-2072-41f8-8e0a-3d9fa59e476c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:08.890614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac39f9-2072-41f8-8e0a-3d9fa59e476c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:08.890818 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac39f9-2072-41f8-8e0a-3d9fa59e476c None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:08.892122 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac39f9-2072-41f8-8e0a-3d9fa59e476c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:08.892506 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac39f9-2072-41f8-8e0a-3d9fa59e476c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:08.894081 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac39f9-2072-41f8-8e0a-3d9fa59e476c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:08.907750 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 622/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:21:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:21:08.914271 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850003b6-4e0b-4720-a777-68620b33376b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:09.042560 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850003b6-4e0b-4720-a777-68620b33376b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:09.080458 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e545e99-fbd5-4a83-b6ca-e41e61c6cf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:09.081232 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e545e99-fbd5-4a83-b6ca-e41e61c6cf0d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:09.081671 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e545e99-fbd5-4a83-b6ca-e41e61c6cf0d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:09.082047 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e545e99-fbd5-4a83-b6ca-e41e61c6cf0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:09.083081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e545e99-fbd5-4a83-b6ca-e41e61c6cf0d None admin] RBAC: Adding query filter params (role.id=785cca921bad4984a86e45ce6c5601c2, scope.project.id=13fba33c46cf40bbb15a08c690b4c98c, user.id=e2244655ba304b9c980b5e68772eefc0) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:09.083501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e545e99-fbd5-4a83-b6ca-e41e61c6cf0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:09.085354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e545e99-fbd5-4a83-b6ca-e41e61c6cf0d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:09.092190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 624/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:21:08 2022] GET /identity/v3/role_assignments?user.id=e2244655ba304b9c980b5e68772eefc0&scope.project.id=13fba33c46cf40bbb15a08c690b4c98c&role.id=785cca921bad4984a86e45ce6c5601c2 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:11.398550 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbbb31-0b11-4cd0-8ccc-939e934d7f97 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:11.399107 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbbb31-0b11-4cd0-8ccc-939e934d7f97 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:11.399882 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbbb31-0b11-4cd0-8ccc-939e934d7f97 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:11.401292 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 623/1247] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:21:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:21:11.406711 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82779b-52d5-49ab-bf13-de1a7909605d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:11.407145 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82779b-52d5-49ab-bf13-de1a7909605d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:11.407236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82779b-52d5-49ab-bf13-de1a7909605d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:11.414812 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:11.430252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ee82779b-52d5-49ab-bf13-de1a7909605d None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:21:11.431474 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ee82779b-52d5-49ab-bf13-de1a7909605d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:21:11.509643 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 625/1248] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:21:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:21:11.533609 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfbbb31-0b11-4cd0-8ccc-939e934d7f97 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:11.553274 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:11.646561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfbbb31-0b11-4cd0-8ccc-939e934d7f97 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:11.686503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a831d5d-31f6-4179-b1de-ebd8338c90f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:11.687268 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a831d5d-31f6-4179-b1de-ebd8338c90f0 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:11.687428 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a831d5d-31f6-4179-b1de-ebd8338c90f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:11.687704 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a831d5d-31f6-4179-b1de-ebd8338c90f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:11.700520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a831d5d-31f6-4179-b1de-ebd8338c90f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:21:11.707431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-3a831d5d-31f6-4179-b1de-ebd8338c90f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:21:11.707431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:11.707431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:11.707431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:11.707431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:11.707431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:11.707431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:11.707431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:11.707431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:11.707431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:11.707431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:11.707431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:11.707431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:21:11.707431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:21:11.707431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:21:11.707431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:21:11.708827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:21:11.708827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:21:11.708827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:21:11.708827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:21:11.708827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:11.708827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:11.708827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:11.708827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:11.708827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:11.708827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:11.708827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:11.708827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:11.708827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:11.708827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:11.708827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:11.708827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:11.714907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:11.714907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:11.714907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:11.714907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:11.714907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:21:11.714907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:21:11.714907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:21:11.714907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:21:11.714907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:21:11.714907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:21:11.714907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:21:11.714907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:21:11.714907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 624/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:21:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:21:11.715787 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82779b-52d5-49ab-bf13-de1a7909605d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:11.829554 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82779b-52d5-49ab-bf13-de1a7909605d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:11.867120 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:11.873779 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b4044-11e9-43b5-b325-7b10f4246faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:11.874757 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b4044-11e9-43b5-b325-7b10f4246faa None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:11.875096 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b4044-11e9-43b5-b325-7b10f4246faa None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:11.875491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b4044-11e9-43b5-b325-7b10f4246faa None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:11.876507 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b4044-11e9-43b5-b325-7b10f4246faa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:11.876987 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b4044-11e9-43b5-b325-7b10f4246faa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:11.879203 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b4044-11e9-43b5-b325-7b10f4246faa None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:11.889551 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 626/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:21:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:11.895990 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a831d5d-31f6-4179-b1de-ebd8338c90f0 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:11.988605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:12.017420 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a831d5d-31f6-4179-b1de-ebd8338c90f0 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:12.054064 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562362be-ee31-43eb-af1e-02c5bb59d3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:12.054695 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562362be-ee31-43eb-af1e-02c5bb59d3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:12.054970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562362be-ee31-43eb-af1e-02c5bb59d3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:12.055242 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562362be-ee31-43eb-af1e-02c5bb59d3f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:12.066804 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562362be-ee31-43eb-af1e-02c5bb59d3f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:12.068446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562362be-ee31-43eb-af1e-02c5bb59d3f8 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:12.084826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-562362be-ee31-43eb-af1e-02c5bb59d3f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:21:12.084826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:12.084826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:12.084826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:12.084826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:12.084826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:12.084826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:12.084826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:12.084826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:12.084826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:12.084826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:12.084826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:12.084826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:21:12.084826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:21:12.084826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:21:12.085780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:21:12.085780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:12.085780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:12.085780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:12.085780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:12.085780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:12.085780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:12.085780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:12.085780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:12.085780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:12.085780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:12.085780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:12.085780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:12.085780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:12.085780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:12.085780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:12.085780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:12.086677 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:21:12.086677 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:21:12.086677 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:21:12.086677 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:21:12.086677 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 625/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:21:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 06 12:21:12.092194 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b4044-11e9-43b5-b325-7b10f4246faa None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:12.201195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b4044-11e9-43b5-b325-7b10f4246faa None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:12.241603 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76941f74-3832-4992-b3c6-c4b7be4499a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:12.242137 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76941f74-3832-4992-b3c6-c4b7be4499a5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:12.242449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76941f74-3832-4992-b3c6-c4b7be4499a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:12.242717 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76941f74-3832-4992-b3c6-c4b7be4499a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:12.243368 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76941f74-3832-4992-b3c6-c4b7be4499a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:12.243656 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76941f74-3832-4992-b3c6-c4b7be4499a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:12.245048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76941f74-3832-4992-b3c6-c4b7be4499a5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:12.257485 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 627/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:21:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:12.263357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562362be-ee31-43eb-af1e-02c5bb59d3f8 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:12.368378 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562362be-ee31-43eb-af1e-02c5bb59d3f8 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:12.395640 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:12.404761 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384968b-f870-476e-8aba-10c7e764cb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:12.405454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384968b-f870-476e-8aba-10c7e764cb92 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:12.405786 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384968b-f870-476e-8aba-10c7e764cb92 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:12.406093 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384968b-f870-476e-8aba-10c7e764cb92 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:12.416025 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384968b-f870-476e-8aba-10c7e764cb92 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:12.417920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384968b-f870-476e-8aba-10c7e764cb92 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:12.433427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-7384968b-f870-476e-8aba-10c7e764cb92 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 12:21:12.433427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:12.433427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:12.433427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:12.433427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:12.433427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:12.433427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:12.433427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:12.433427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:12.433427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:12.433427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:12.433427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:12.433427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:21:12.433427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:21:12.433427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:21:12.433427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:21:12.434797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:12.434797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:12.434797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:21:12.434797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:21:12.434797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:21:12.434797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:21:12.434797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:12.434797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:12.434797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:12.434797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:12.434797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:12.434797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:12.434797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:12.434797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:12.434797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:12.434797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:12.434797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:12.434797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:12.436044 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:12.436044 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:12.436044 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:21:12.436044 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:21:12.436044 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:21:12.436044 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:21:12.436044 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:21:12.436044 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:21:12.436044 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 12:21:12.436044 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:21:12.436044 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 626/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:21:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:21:12.443296 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76941f74-3832-4992-b3c6-c4b7be4499a5 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:12.505754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:12.554213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76941f74-3832-4992-b3c6-c4b7be4499a5 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:12.593473 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091d0c5-9a56-4a92-98df-ae14699f32ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:12.594119 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091d0c5-9a56-4a92-98df-ae14699f32ab None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:12.594399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091d0c5-9a56-4a92-98df-ae14699f32ab None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:12.594674 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091d0c5-9a56-4a92-98df-ae14699f32ab None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:12.595462 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091d0c5-9a56-4a92-98df-ae14699f32ab None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:12.595768 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091d0c5-9a56-4a92-98df-ae14699f32ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:12.597244 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091d0c5-9a56-4a92-98df-ae14699f32ab None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:12.617720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 628/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Sat Aug 6 12:21:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:12.624005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384968b-f870-476e-8aba-10c7e764cb92 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:12.747973 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384968b-f870-476e-8aba-10c7e764cb92 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:12.789396 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092621da-057e-49d1-9aa5-00862237cf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:12.790070 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092621da-057e-49d1-9aa5-00862237cf32 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:12.790415 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092621da-057e-49d1-9aa5-00862237cf32 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:12.790687 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092621da-057e-49d1-9aa5-00862237cf32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:12.847897 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:12.861496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092621da-057e-49d1-9aa5-00862237cf32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:12.862947 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092621da-057e-49d1-9aa5-00862237cf32 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:12.962861 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 627/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:21:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 12:21:12.969467 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091d0c5-9a56-4a92-98df-ae14699f32ab None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:13.088273 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091d0c5-9a56-4a92-98df-ae14699f32ab None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:13.104179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:13.130677 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76753e7e-475a-4016-b002-864e8ffb2c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:13.131309 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76753e7e-475a-4016-b002-864e8ffb2c87 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:13.131583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76753e7e-475a-4016-b002-864e8ffb2c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:13.131850 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76753e7e-475a-4016-b002-864e8ffb2c87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:13.141997 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76753e7e-475a-4016-b002-864e8ffb2c87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:13.143420 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76753e7e-475a-4016-b002-864e8ffb2c87 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:13.161398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-76753e7e-475a-4016-b002-864e8ffb2c87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:21:13.161398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:13.161398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:13.161398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:13.161398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:13.161398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:13.161398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:13.161398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:13.161398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:13.161398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:13.161398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:13.161398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:13.161398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:21:13.161398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:21:13.161398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:21:13.162615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:21:13.162615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:13.162615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:13.162615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:13.162615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:13.162615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:13.162615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:13.162615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:13.162615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:13.162615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:13.162615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:13.162615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:13.162615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:13.162615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:13.162615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:13.162615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:13.162615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:13.168226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:21:13.168226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:21:13.168226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:21:13.168226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:21:13.168226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 629/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:21:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:21:13.168730 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092621da-057e-49d1-9aa5-00862237cf32 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:13.260150 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:13.274874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092621da-057e-49d1-9aa5-00862237cf32 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:13.312762 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391b6a3-ff3f-470a-93b1-0f805262646a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:13.313375 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391b6a3-ff3f-470a-93b1-0f805262646a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:13.313704 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391b6a3-ff3f-470a-93b1-0f805262646a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:13.314119 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391b6a3-ff3f-470a-93b1-0f805262646a None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:13.314890 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f391b6a3-ff3f-470a-93b1-0f805262646a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:13.315273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f391b6a3-ff3f-470a-93b1-0f805262646a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:13.316840 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f391b6a3-ff3f-470a-93b1-0f805262646a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:13.330428 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 628/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:13.337001 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76753e7e-475a-4016-b002-864e8ffb2c87 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:13.465891 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76753e7e-475a-4016-b002-864e8ffb2c87 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:13.503209 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73f404-cfe6-4af0-99b0-79f00b3f05f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:13.503845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73f404-cfe6-4af0-99b0-79f00b3f05f9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:13.504119 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73f404-cfe6-4af0-99b0-79f00b3f05f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:13.504421 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73f404-cfe6-4af0-99b0-79f00b3f05f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:13.511246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf73f404-cfe6-4af0-99b0-79f00b3f05f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:13.512699 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf73f404-cfe6-4af0-99b0-79f00b3f05f9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:13.526058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-bf73f404-cfe6-4af0-99b0-79f00b3f05f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:21:13.526058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:13.526058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:13.526058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:13.526058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:13.526058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:13.526058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:13.526058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:13.526058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:13.526058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:13.526058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:13.526058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:13.526058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:21:13.526058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:21:13.526058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:21:13.527163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:21:13.527163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:13.527163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:13.527163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:13.527163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:13.527163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:13.527163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:13.527163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:13.527163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:13.527163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:13.527163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:13.527163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:13.527163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:13.527163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:13.527163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:13.527163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:13.527163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:13.528199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:21:13.528199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:21:13.528199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:21:13.528199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:21:13.528199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:21:13.528199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:21:13.528199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:21:13.528199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:21:13.528199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 630/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:21:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:21:13.534531 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391b6a3-ff3f-470a-93b1-0f805262646a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:13.642170 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391b6a3-ff3f-470a-93b1-0f805262646a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:13.691035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271c42a-0e20-4302-bccb-ff3651179131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:13.692074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271c42a-0e20-4302-bccb-ff3651179131 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:13.692532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271c42a-0e20-4302-bccb-ff3651179131 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:13.692961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271c42a-0e20-4302-bccb-ff3651179131 None admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:13.694200 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271c42a-0e20-4302-bccb-ff3651179131 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:13.694559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271c42a-0e20-4302-bccb-ff3651179131 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:13.696330 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271c42a-0e20-4302-bccb-ff3651179131 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:13.713550 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 629/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:21:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:13.720710 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73f404-cfe6-4af0-99b0-79f00b3f05f9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:13.751286 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:13.831164 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73f404-cfe6-4af0-99b0-79f00b3f05f9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:13.867876 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92ff42-718f-425f-a768-ac153eed11a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:13.868529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92ff42-718f-425f-a768-ac153eed11a3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:13.868806 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92ff42-718f-425f-a768-ac153eed11a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:13.869078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92ff42-718f-425f-a768-ac153eed11a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:13.869877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92ff42-718f-425f-a768-ac153eed11a3 None admin] RBAC: Adding query filter params (role.id=144c8425688b409384e1e8ed8279d88e, scope.project.id=13fba33c46cf40bbb15a08c690b4c98c, user.id=e2244655ba304b9c980b5e68772eefc0) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:13.870186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92ff42-718f-425f-a768-ac153eed11a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:13.871642 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92ff42-718f-425f-a768-ac153eed11a3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:13.878365 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 631/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:21:13 2022] GET /identity/v3/role_assignments?user.id=e2244655ba304b9c980b5e68772eefc0&scope.project.id=13fba33c46cf40bbb15a08c690b4c98c&role.id=144c8425688b409384e1e8ed8279d88e => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:21:15.892429 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01881a66-0e48-4f80-9653-fa9f856f1873 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:15.892953 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01881a66-0e48-4f80-9653-fa9f856f1873 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:15.893073 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01881a66-0e48-4f80-9653-fa9f856f1873 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:15.894342 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 630/1261] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:21:15.899310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dac8d1-df23-40ce-97e4-ce1010d1ae65 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:15.899716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dac8d1-df23-40ce-97e4-ce1010d1ae65 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:15.899851 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dac8d1-df23-40ce-97e4-ce1010d1ae65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:15.918021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e9dac8d1-df23-40ce-97e4-ce1010d1ae65 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:21:15.919184 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e9dac8d1-df23-40ce-97e4-ce1010d1ae65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:21:15.982576 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 632/1262] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:21:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:21:16.005146 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01881a66-0e48-4f80-9653-fa9f856f1873 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:16.017833 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:16.108801 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01881a66-0e48-4f80-9653-fa9f856f1873 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:16.141892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b3d3c-0853-4ef1-a834-f3ef3f2d136b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:16.142561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b3d3c-0853-4ef1-a834-f3ef3f2d136b None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:16.142788 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b3d3c-0853-4ef1-a834-f3ef3f2d136b None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:16.143057 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b3d3c-0853-4ef1-a834-f3ef3f2d136b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:16.152764 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b3d3c-0853-4ef1-a834-f3ef3f2d136b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:21:16.157731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-da5b3d3c-0853-4ef1-a834-f3ef3f2d136b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:21:16.157731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:16.157731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:16.157731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:16.157731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:16.157731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:16.157731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:16.157731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:16.157731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:16.157731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:16.157731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:16.157731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:16.157731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:21:16.157731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:21:16.157731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:21:16.157731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:21:16.158820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:21:16.158820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:21:16.158820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:21:16.158820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:21:16.158820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:16.158820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:16.158820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:16.158820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:16.158820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:16.158820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:16.158820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:16.158820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:16.158820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:16.158820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:16.158820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:16.158820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:16.159858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:16.159858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:16.159858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:16.159858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:16.159858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:21:16.159858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:21:16.159858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:21:16.159858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:21:16.159858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:21:16.159858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:21:16.159858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:21:16.159858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:21:16.159858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 631/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:21:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:21:16.164846 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dac8d1-df23-40ce-97e4-ce1010d1ae65 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:16.183088 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:16.260959 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dac8d1-df23-40ce-97e4-ce1010d1ae65 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:16.293049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efc58c-3bcf-4a83-8773-271682c371d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:16.293655 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efc58c-3bcf-4a83-8773-271682c371d2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:16.293937 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efc58c-3bcf-4a83-8773-271682c371d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:16.294210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efc58c-3bcf-4a83-8773-271682c371d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:16.294917 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89efc58c-3bcf-4a83-8773-271682c371d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:16.295221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89efc58c-3bcf-4a83-8773-271682c371d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:16.296611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89efc58c-3bcf-4a83-8773-271682c371d2 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:16.305055 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 633/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:21:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:16.310971 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b3d3c-0853-4ef1-a834-f3ef3f2d136b None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:16.376208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:16.407350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b3d3c-0853-4ef1-a834-f3ef3f2d136b None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:16.440170 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28eee41-9f2a-400d-8dbe-7688ca3895de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:16.440808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28eee41-9f2a-400d-8dbe-7688ca3895de None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:16.441084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28eee41-9f2a-400d-8dbe-7688ca3895de None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:16.441353 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28eee41-9f2a-400d-8dbe-7688ca3895de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:16.450916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28eee41-9f2a-400d-8dbe-7688ca3895de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:16.452377 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28eee41-9f2a-400d-8dbe-7688ca3895de None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:16.467051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-f28eee41-9f2a-400d-8dbe-7688ca3895de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:21:16.467051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:16.467051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:16.467051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:16.467051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:16.467051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:16.467051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:16.467051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:16.467051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:16.467051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:16.467051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:16.467051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:16.467051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:21:16.467051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:21:16.467051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:21:16.468084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:21:16.468084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:16.468084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:16.468084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:16.468084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:16.468084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:16.468084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:16.468084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:16.468084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:16.468084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:16.468084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:16.468084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:16.468084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:16.468084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:16.468084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:16.468084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:16.468084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:16.470271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:21:16.470271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:21:16.470271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:21:16.470271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:21:16.470271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 632/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:21:16.474306 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efc58c-3bcf-4a83-8773-271682c371d2 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:16.553153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:16.572138 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efc58c-3bcf-4a83-8773-271682c371d2 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:16.604500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9bfba-3339-4909-9425-509bb345f582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:16.605095 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9bfba-3339-4909-9425-509bb345f582 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:16.605374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9bfba-3339-4909-9425-509bb345f582 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:16.605644 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9bfba-3339-4909-9425-509bb345f582 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:16.606323 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9bfba-3339-4909-9425-509bb345f582 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:16.606623 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9bfba-3339-4909-9425-509bb345f582 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:16.607962 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9bfba-3339-4909-9425-509bb345f582 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:16.620087 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 634/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:16.625559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28eee41-9f2a-400d-8dbe-7688ca3895de None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:16.733048 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28eee41-9f2a-400d-8dbe-7688ca3895de None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:16.752982 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:16.769720 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997164c-de63-4bfd-93fb-84251be1ad76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:16.770358 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997164c-de63-4bfd-93fb-84251be1ad76 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:16.770638 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997164c-de63-4bfd-93fb-84251be1ad76 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:16.770926 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997164c-de63-4bfd-93fb-84251be1ad76 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:16.778484 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997164c-de63-4bfd-93fb-84251be1ad76 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:16.779918 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997164c-de63-4bfd-93fb-84251be1ad76 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:16.791512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-0997164c-de63-4bfd-93fb-84251be1ad76 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 12:21:16.791512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:16.791512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:16.791512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:16.791512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:16.791512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:16.791512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:16.791512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:16.791512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:16.791512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:16.791512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:16.791512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:16.791512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:21:16.791512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:21:16.791512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:21:16.791512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:21:16.792606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:16.792606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:16.792606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:21:16.792606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:21:16.792606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:21:16.792606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:21:16.792606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:16.792606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:16.792606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:16.792606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:16.792606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:16.792606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:16.792606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:16.792606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:16.792606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:16.792606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:16.792606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:16.792606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:16.793789 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:16.793789 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:16.793789 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:21:16.793789 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:21:16.793789 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:21:16.793789 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:21:16.793789 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:21:16.793789 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:21:16.793789 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 12:21:16.793789 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:21:16.793789 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 633/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:21:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:21:16.799138 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9bfba-3339-4909-9425-509bb345f582 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:16.894949 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9bfba-3339-4909-9425-509bb345f582 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:16.927595 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7437de-a446-497a-b5be-d3f626d07aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:16.928200 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7437de-a446-497a-b5be-d3f626d07aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:16.928516 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7437de-a446-497a-b5be-d3f626d07aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:16.928795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7437de-a446-497a-b5be-d3f626d07aa7 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:16.929581 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7437de-a446-497a-b5be-d3f626d07aa7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:16.929888 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7437de-a446-497a-b5be-d3f626d07aa7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:16.931308 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7437de-a446-497a-b5be-d3f626d07aa7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:16.939669 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:16.948921 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 635/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Sat Aug 6 12:21:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:16.954957 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997164c-de63-4bfd-93fb-84251be1ad76 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:17.051864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997164c-de63-4bfd-93fb-84251be1ad76 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:17.084023 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9f6b8-cee7-4523-b85e-f39a19b2e457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:17.084671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9f6b8-cee7-4523-b85e-f39a19b2e457 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:17.084946 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9f6b8-cee7-4523-b85e-f39a19b2e457 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:17.085213 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9f6b8-cee7-4523-b85e-f39a19b2e457 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:17.095829 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c9f6b8-cee7-4523-b85e-f39a19b2e457 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:17.097289 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c9f6b8-cee7-4523-b85e-f39a19b2e457 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:17.116476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-85c9f6b8-cee7-4523-b85e-f39a19b2e457 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:21:17.116476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:17.116476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:17.116476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:17.116476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:17.116476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:17.116476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:17.116476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:17.116476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:17.116476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:17.116476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:17.116476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:17.116476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:21:17.116476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:21:17.116476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:21:17.117613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:21:17.117613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:17.117613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:17.117613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:17.117613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:17.117613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:17.117613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:17.117613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:17.117613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:17.117613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:17.117613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:17.117613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:17.117613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:17.117613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:17.117613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:17.117613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:17.117613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:17.118741 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:21:17.118741 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:21:17.118741 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:21:17.118741 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:21:17.118741 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 634/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:21:17.123351 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7437de-a446-497a-b5be-d3f626d07aa7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:17.218387 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7437de-a446-497a-b5be-d3f626d07aa7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:17.251853 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41580d0b-ec24-49be-82b5-58c80a973e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:17.252416 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41580d0b-ec24-49be-82b5-58c80a973e8d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:17.252696 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41580d0b-ec24-49be-82b5-58c80a973e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:17.252960 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41580d0b-ec24-49be-82b5-58c80a973e8d None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:17.253656 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41580d0b-ec24-49be-82b5-58c80a973e8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:17.253949 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41580d0b-ec24-49be-82b5-58c80a973e8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:17.255283 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41580d0b-ec24-49be-82b5-58c80a973e8d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:17.266623 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 636/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:17.271939 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9f6b8-cee7-4523-b85e-f39a19b2e457 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:17.301987 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:17.369374 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9f6b8-cee7-4523-b85e-f39a19b2e457 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:17.402192 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae78ec2a-bc6a-45e8-87bf-8e61f4fc14fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:17.402809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78ec2a-bc6a-45e8-87bf-8e61f4fc14fc None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:17.403091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78ec2a-bc6a-45e8-87bf-8e61f4fc14fc None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:17.403365 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78ec2a-bc6a-45e8-87bf-8e61f4fc14fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:17.409851 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae78ec2a-bc6a-45e8-87bf-8e61f4fc14fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:17.411293 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae78ec2a-bc6a-45e8-87bf-8e61f4fc14fc None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:17.422383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-ae78ec2a-bc6a-45e8-87bf-8e61f4fc14fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:21:17.422383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:17.422383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:17.422383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:17.422383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:17.422383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:17.422383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:17.422383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:17.422383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:17.422383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:17.422383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:17.422383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:17.422383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:21:17.422383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:21:17.422383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:21:17.423401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:21:17.423401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:17.423401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:17.423401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:17.423401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:17.423401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:17.423401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:17.423401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:17.423401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:17.423401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:17.423401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:17.423401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:17.423401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:17.423401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:17.423401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:17.423401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:17.423401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:17.424411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:21:17.424411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:21:17.424411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:21:17.424411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:21:17.424411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:21:17.424411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:21:17.424411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:21:17.424411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:21:17.424411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 635/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:21:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:21:17.429189 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41580d0b-ec24-49be-82b5-58c80a973e8d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:17.485329 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:17.525975 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41580d0b-ec24-49be-82b5-58c80a973e8d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:17.557140 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399746ed-3c5f-449b-a96f-8d4d4ddcfca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:17.557735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399746ed-3c5f-449b-a96f-8d4d4ddcfca7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:17.558016 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399746ed-3c5f-449b-a96f-8d4d4ddcfca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:17.558287 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399746ed-3c5f-449b-a96f-8d4d4ddcfca7 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:17.559025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399746ed-3c5f-449b-a96f-8d4d4ddcfca7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:17.559326 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399746ed-3c5f-449b-a96f-8d4d4ddcfca7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:17.560753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399746ed-3c5f-449b-a96f-8d4d4ddcfca7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:17.572683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 637/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:21:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:21:17.578985 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae78ec2a-bc6a-45e8-87bf-8e61f4fc14fc None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:17.677914 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:17.689079 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae78ec2a-bc6a-45e8-87bf-8e61f4fc14fc None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:17.721262 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b8676-f088-4de6-9400-26503d43fc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:17.721916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b8676-f088-4de6-9400-26503d43fc21 None admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:17.722195 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b8676-f088-4de6-9400-26503d43fc21 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:17.722468 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b8676-f088-4de6-9400-26503d43fc21 None admin] PATH_INFO: `/v3/projects/13fba33c46cf40bbb15a08c690b4c98c/users/e2244655ba304b9c980b5e68772eefc0/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:17.724377 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b8676-f088-4de6-9400-26503d43fc21 None admin] RBAC: Authorizing `identity:create_grant(project_id=13fba33c46cf40bbb15a08c690b4c98c, user_id=e2244655ba304b9c980b5e68772eefc0, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:17.725799 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b8676-f088-4de6-9400-26503d43fc21 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:17.739530 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 636/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Sat Aug 6 12:21:17 2022] PUT /identity/v3/projects/13fba33c46cf40bbb15a08c690b4c98c/users/e2244655ba304b9c980b5e68772eefc0/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:21:19.700544 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b929f4-fe42-4f80-9291-8797538e422c None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:19.701082 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b929f4-fe42-4f80-9291-8797538e422c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:19.701082 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b929f4-fe42-4f80-9291-8797538e422c None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:19.702430 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 638/1274] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:21:19.707422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078211a-1d2e-4a66-9127-0a98d8da1126 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:19.707849 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078211a-1d2e-4a66-9127-0a98d8da1126 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:19.707958 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078211a-1d2e-4a66-9127-0a98d8da1126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:19.726086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-0078211a-1d2e-4a66-9127-0a98d8da1126 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:21:19.727249 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0078211a-1d2e-4a66-9127-0a98d8da1126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:21:19.792028 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 637/1275] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:21:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:21:19.815509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b929f4-fe42-4f80-9291-8797538e422c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:19.913688 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b929f4-fe42-4f80-9291-8797538e422c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:19.930852 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:19.946990 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c970cf5-69d1-4b12-9a22-f83430b4e3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:19.947609 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c970cf5-69d1-4b12-9a22-f83430b4e3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:19.947888 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c970cf5-69d1-4b12-9a22-f83430b4e3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:19.948161 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c970cf5-69d1-4b12-9a22-f83430b4e3c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:19.958428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c970cf5-69d1-4b12-9a22-f83430b4e3c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 06 12:21:19.963226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-2c970cf5-69d1-4b12-9a22-f83430b4e3c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:21:19.963226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:19.963226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:19.963226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:19.963226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:19.963226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:19.963226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:19.963226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:19.963226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:19.963226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:19.963226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:19.963226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:19.963226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 06 12:21:19.963226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 06 12:21:19.963226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 06 12:21:19.963226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 06 12:21:19.964328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 06 12:21:19.964328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 06 12:21:19.964328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 06 12:21:19.964328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 06 12:21:19.964328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:19.964328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:19.964328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:19.964328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:19.964328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:19.964328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:19.964328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:19.964328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:19.964328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:19.964328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:19.964328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:19.964328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:19.965188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:19.965188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:19.965188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:19.965188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:19.965188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 06 12:21:19.965188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 06 12:21:19.965188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 06 12:21:19.965188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 06 12:21:19.965188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 06 12:21:19.965188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 06 12:21:19.965188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 06 12:21:19.965188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:21:19.965188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 639/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Sat Aug 6 12:21:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:21:19.970873 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078211a-1d2e-4a66-9127-0a98d8da1126 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:20.047405 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:20.065279 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078211a-1d2e-4a66-9127-0a98d8da1126 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:20.097908 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853112bd-f807-4c71-9b0b-3e0666dbb5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:20.098518 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853112bd-f807-4c71-9b0b-3e0666dbb5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:20.098793 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853112bd-f807-4c71-9b0b-3e0666dbb5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:20.099064 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853112bd-f807-4c71-9b0b-3e0666dbb5d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:20.099761 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853112bd-f807-4c71-9b0b-3e0666dbb5d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:20.100060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853112bd-f807-4c71-9b0b-3e0666dbb5d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:20.101466 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853112bd-f807-4c71-9b0b-3e0666dbb5d2 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:20.109355 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 638/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Sat Aug 6 12:21:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:20.115414 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c970cf5-69d1-4b12-9a22-f83430b4e3c8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:20.211558 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c970cf5-69d1-4b12-9a22-f83430b4e3c8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:20.244617 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f1d87-dfc2-459f-88ee-5945dbd06574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:20.245239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f1d87-dfc2-459f-88ee-5945dbd06574 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:20.245518 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f1d87-dfc2-459f-88ee-5945dbd06574 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:20.245789 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f1d87-dfc2-459f-88ee-5945dbd06574 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:20.257068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6f1d87-dfc2-459f-88ee-5945dbd06574 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:20.258504 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6f1d87-dfc2-459f-88ee-5945dbd06574 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:20.272930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-3e6f1d87-dfc2-459f-88ee-5945dbd06574 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:21:20.272930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:20.272930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:20.272930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:20.272930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:20.272930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:20.272930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:20.272930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:20.272930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:20.272930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:20.272930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:20.272930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:20.272930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:21:20.272930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:21:20.272930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:21:20.273947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:21:20.273947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:20.273947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:20.273947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:20.273947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:20.273947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:20.273947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:20.273947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:20.273947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:20.273947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:20.273947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:20.273947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:20.273947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:20.273947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:20.273947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:20.273947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:20.273947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:20.274895 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:21:20.274895 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:21:20.274895 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:21:20.274895 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:21:20.274895 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 640/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:21:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:21:20.279990 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853112bd-f807-4c71-9b0b-3e0666dbb5d2 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:20.374645 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853112bd-f807-4c71-9b0b-3e0666dbb5d2 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:20.406885 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d340ff-fd18-4b7a-8f18-5ded864c2085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:20.407472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d340ff-fd18-4b7a-8f18-5ded864c2085 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:20.407746 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d340ff-fd18-4b7a-8f18-5ded864c2085 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:20.408014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d340ff-fd18-4b7a-8f18-5ded864c2085 None admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:20.408712 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d340ff-fd18-4b7a-8f18-5ded864c2085 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:20.409010 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d340ff-fd18-4b7a-8f18-5ded864c2085 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:20.410360 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d340ff-fd18-4b7a-8f18-5ded864c2085 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:20.421382 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 639/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:20.426855 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f1d87-dfc2-459f-88ee-5945dbd06574 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:20.457646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:20.525319 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f1d87-dfc2-459f-88ee-5945dbd06574 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:20.556971 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4da97-177d-41f8-8de9-a8cab815144d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:20.557586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4da97-177d-41f8-8de9-a8cab815144d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:20.557861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4da97-177d-41f8-8de9-a8cab815144d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:20.558129 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4da97-177d-41f8-8de9-a8cab815144d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:20.565798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4da97-177d-41f8-8de9-a8cab815144d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:20.567205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4da97-177d-41f8-8de9-a8cab815144d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:20.578190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-24b4da97-177d-41f8-8de9-a8cab815144d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 06 12:21:20.578190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:20.578190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:20.578190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:20.578190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:20.578190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:20.578190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:20.578190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:20.578190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:20.578190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:20.578190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:20.578190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:20.578190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 06 12:21:20.578190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 06 12:21:20.578190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 06 12:21:20.578190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 06 12:21:20.579405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:20.579405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:20.579405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 06 12:21:20.579405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:21:20.579405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 06 12:21:20.579405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 06 12:21:20.579405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:20.579405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:20.579405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:20.579405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:20.579405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:20.579405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:20.579405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:20.579405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:20.579405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:20.579405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:20.579405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:20.579405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:20.580753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:20.580753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:20.580753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 06 12:21:20.580753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 06 12:21:20.580753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 06 12:21:20.580753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 06 12:21:20.580753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 06 12:21:20.580753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 06 12:21:20.580753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 06 12:21:20.580753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:21:20.580753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 641/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:21:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:21:20.584982 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d340ff-fd18-4b7a-8f18-5ded864c2085 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:20.588619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:20.694437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d340ff-fd18-4b7a-8f18-5ded864c2085 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:20.730863 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2621f95-ee39-4e78-b09f-53ba4dc1039a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:20.731477 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2621f95-ee39-4e78-b09f-53ba4dc1039a None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:20.731725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2621f95-ee39-4e78-b09f-53ba4dc1039a None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:20.731990 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2621f95-ee39-4e78-b09f-53ba4dc1039a None admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:20.732799 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2621f95-ee39-4e78-b09f-53ba4dc1039a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:20.733100 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2621f95-ee39-4e78-b09f-53ba4dc1039a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:20.735051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2621f95-ee39-4e78-b09f-53ba4dc1039a None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:20.751788 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 640/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Sat Aug 6 12:21:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:20.757646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4da97-177d-41f8-8de9-a8cab815144d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:20.845197 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:20.854559 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4da97-177d-41f8-8de9-a8cab815144d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:20.886110 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1236f2-c907-4463-b76a-b7ba7f8aa0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:20.886702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1236f2-c907-4463-b76a-b7ba7f8aa0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:20.886978 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1236f2-c907-4463-b76a-b7ba7f8aa0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:20.887247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1236f2-c907-4463-b76a-b7ba7f8aa0a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:20.951635 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1236f2-c907-4463-b76a-b7ba7f8aa0a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:20.953127 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1236f2-c907-4463-b76a-b7ba7f8aa0a4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:21.039491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:21.050352 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 642/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Sat Aug 6 12:21:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 06 12:21:21.056024 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2621f95-ee39-4e78-b09f-53ba4dc1039a None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:21.114693 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:21.151486 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2621f95-ee39-4e78-b09f-53ba4dc1039a None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:21.185046 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d691e-036b-4b77-b3de-e9f8e360a4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:21.185666 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d691e-036b-4b77-b3de-e9f8e360a4ad None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:21.185944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d691e-036b-4b77-b3de-e9f8e360a4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:21.186215 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d691e-036b-4b77-b3de-e9f8e360a4ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:21.195761 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d691e-036b-4b77-b3de-e9f8e360a4ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:21.197241 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d691e-036b-4b77-b3de-e9f8e360a4ad None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:21.211357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-d88d691e-036b-4b77-b3de-e9f8e360a4ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:21:21.211357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:21.211357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:21.211357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:21.211357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:21.211357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:21.211357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:21.211357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:21.211357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:21.211357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:21.211357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:21.211357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:21.211357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 06 12:21:21.211357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 06 12:21:21.211357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 06 12:21:21.212432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 06 12:21:21.212432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:21.212432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:21.212432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:21.212432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:21.212432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:21.212432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:21.212432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:21.212432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:21.212432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:21.212432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:21.212432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:21.212432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:21.212432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:21.212432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:21.212432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:21.212432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:21.213743 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 06 12:21:21.213743 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 06 12:21:21.213743 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 06 12:21:21.213743 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:21:21.213743 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 641/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:21:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:21:21.218112 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1236f2-c907-4463-b76a-b7ba7f8aa0a4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:21.313908 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1236f2-c907-4463-b76a-b7ba7f8aa0a4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:21.346132 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc36d9-bc8d-49cc-a1f8-4f038ad37ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:21.346727 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc36d9-bc8d-49cc-a1f8-4f038ad37ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:21.347001 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc36d9-bc8d-49cc-a1f8-4f038ad37ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:21.347269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc36d9-bc8d-49cc-a1f8-4f038ad37ff3 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:21.347942 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc36d9-bc8d-49cc-a1f8-4f038ad37ff3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:21.348243 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc36d9-bc8d-49cc-a1f8-4f038ad37ff3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:21.349613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc36d9-bc8d-49cc-a1f8-4f038ad37ff3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:21.360322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 643/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Sat Aug 6 12:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:21.366399 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d691e-036b-4b77-b3de-e9f8e360a4ad None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:21.461063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d691e-036b-4b77-b3de-e9f8e360a4ad None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:21.474398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:21.494027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b9f69-2ff0-4c9e-9a9a-984e39779753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:21.494639 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b9f69-2ff0-4c9e-9a9a-984e39779753 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:21.494915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b9f69-2ff0-4c9e-9a9a-984e39779753 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:21.495184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b9f69-2ff0-4c9e-9a9a-984e39779753 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:21.501628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b9f69-2ff0-4c9e-9a9a-984e39779753 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:21.503051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b9f69-2ff0-4c9e-9a9a-984e39779753 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:21.514074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application [None req-2d0b9f69-2ff0-4c9e-9a9a-984e39779753 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:21:21.514074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:21.514074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:21.514074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:21.514074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:21.514074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:21.514074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:21.514074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:21.514074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:21.514074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:21.514074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:21.514074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:21.514074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 06 12:21:21.514074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 06 12:21:21.514074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 06 12:21:21.515106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 06 12:21:21.515106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:21.515106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:21.515106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:21.515106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:21.515106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:21.515106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:21.515106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:21.515106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:21.515106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:21.515106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:21.515106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:21.515106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:21.515106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:21.515106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:21.515106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:21.515106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:21.516923 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 06 12:21:21.516923 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 06 12:21:21.516923 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 06 12:21:21.516923 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 06 12:21:21.516923 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 06 12:21:21.516923 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 06 12:21:21.516923 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 06 12:21:21.516923 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: ERROR keystone.server.flask.application Aug 06 12:21:21.516923 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 642/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Sat Aug 6 12:21:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:21:21.521354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc36d9-bc8d-49cc-a1f8-4f038ad37ff3 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:21.571080 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:21.619822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc36d9-bc8d-49cc-a1f8-4f038ad37ff3 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:21.654107 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db04c39-2db4-4c8b-97bd-d754c30a99ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:21.654787 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db04c39-2db4-4c8b-97bd-d754c30a99ee None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:21.655070 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db04c39-2db4-4c8b-97bd-d754c30a99ee None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:21.655340 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db04c39-2db4-4c8b-97bd-d754c30a99ee None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:21.656090 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db04c39-2db4-4c8b-97bd-d754c30a99ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:21.656445 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db04c39-2db4-4c8b-97bd-d754c30a99ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:21.658131 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db04c39-2db4-4c8b-97bd-d754c30a99ee None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:21.671021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 644/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Sat Aug 6 12:21:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:21:21.677605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b9f69-2ff0-4c9e-9a9a-984e39779753 None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:21.774969 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b9f69-2ff0-4c9e-9a9a-984e39779753 None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:21.810446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aca204-13f1-4337-8256-d46ede561bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:21.811050 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aca204-13f1-4337-8256-d46ede561bcf None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:21.811329 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aca204-13f1-4337-8256-d46ede561bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:21.811600 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aca204-13f1-4337-8256-d46ede561bcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:21.812440 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aca204-13f1-4337-8256-d46ede561bcf None admin] RBAC: Adding query filter params (role.id=144c8425688b409384e1e8ed8279d88e, scope.project.id=13fba33c46cf40bbb15a08c690b4c98c, user.id=e2244655ba304b9c980b5e68772eefc0) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:21.812747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aca204-13f1-4337-8256-d46ede561bcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:21.814167 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aca204-13f1-4337-8256-d46ede561bcf None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:21.819786 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 643/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Sat Aug 6 12:21:21 2022] GET /identity/v3/role_assignments?user.id=e2244655ba304b9c980b5e68772eefc0&scope.project.id=13fba33c46cf40bbb15a08c690b4c98c&role.id=144c8425688b409384e1e8ed8279d88e => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:23.798132 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616d6c3-ead5-4a90-8155-167de2a98b92 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:23.798660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616d6c3-ead5-4a90-8155-167de2a98b92 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:23.798660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616d6c3-ead5-4a90-8155-167de2a98b92 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:23.800014 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 645/1288] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:21:23.805127 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554034bb-4995-4922-9f34-9274d475c238 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:23.805512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554034bb-4995-4922-9f34-9274d475c238 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:23.805669 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554034bb-4995-4922-9f34-9274d475c238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:23.823937 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-554034bb-4995-4922-9f34-9274d475c238 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:21:23.825129 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-554034bb-4995-4922-9f34-9274d475c238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:21:23.852828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:23.889374 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 644/1289] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:21:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:21:23.910937 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616d6c3-ead5-4a90-8155-167de2a98b92 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:24.008995 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616d6c3-ead5-4a90-8155-167de2a98b92 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:24.019390 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:24.042507 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bafbb0-b1ed-4bda-a164-96026e78633a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:24.043133 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bafbb0-b1ed-4bda-a164-96026e78633a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:24.043410 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bafbb0-b1ed-4bda-a164-96026e78633a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:24.043681 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bafbb0-b1ed-4bda-a164-96026e78633a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:24.044326 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bafbb0-b1ed-4bda-a164-96026e78633a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:24.045673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bafbb0-b1ed-4bda-a164-96026e78633a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:24.055034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-e9bafbb0-b1ed-4bda-a164-96026e78633a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 12:21:24.055034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:24.055034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:21:24.055034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:21:24.055034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:21:24.055034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:21:24.055034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:21:24.055034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:21:24.055034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 12:21:24.055034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:21:24.055034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:21:24.055034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:21:24.055034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:24.055034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:24.055034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:24.055034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:24.055034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:24.056211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:24.056211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:24.056211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:24.056211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:24.056211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:24.056211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:24.056211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:21:24.056211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:21:24.056211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:21:24.056211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:21:24.056211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:24.056211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:24.056211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:24.056211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:24.056211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:24.056211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:24.057106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:24.057106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:24.057106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:24.057106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:24.057106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:24.057106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:24.057106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:24.057106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:24.057106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:24.057106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:24.057106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:21:24.057106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:21:24.057106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 12:21:24.057106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:21:24.057106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 646/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Sat Aug 6 12:21:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:21:24.062179 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554034bb-4995-4922-9f34-9274d475c238 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:24.155484 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554034bb-4995-4922-9f34-9274d475c238 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:24.188358 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892025b-8d3f-45ff-8ce6-0d797493a28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:24.188957 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892025b-8d3f-45ff-8ce6-0d797493a28e None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:24.189231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892025b-8d3f-45ff-8ce6-0d797493a28e None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:24.189499 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892025b-8d3f-45ff-8ce6-0d797493a28e None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:24.190158 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2892025b-8d3f-45ff-8ce6-0d797493a28e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:24.190455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2892025b-8d3f-45ff-8ce6-0d797493a28e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:24.191792 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2892025b-8d3f-45ff-8ce6-0d797493a28e None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:24.196431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 645/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Sat Aug 6 12:21:24 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:24.202131 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bafbb0-b1ed-4bda-a164-96026e78633a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:24.299341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bafbb0-b1ed-4bda-a164-96026e78633a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:24.331444 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45b749-118c-4fc7-ab7a-9b59c6b8841c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:24.331985 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45b749-118c-4fc7-ab7a-9b59c6b8841c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:24.332264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45b749-118c-4fc7-ab7a-9b59c6b8841c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:24.332568 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45b749-118c-4fc7-ab7a-9b59c6b8841c None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:24.333236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45b749-118c-4fc7-ab7a-9b59c6b8841c None admin] RBAC: Adding query filter params (type=placement) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:24.333538 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45b749-118c-4fc7-ab7a-9b59c6b8841c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:24.334887 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45b749-118c-4fc7-ab7a-9b59c6b8841c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:24.339234 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 647/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Sat Aug 6 12:21:24 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:26.337991 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1a3c1-e0a5-48d5-bc85-e9d17d82829d None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:26.338511 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1a3c1-e0a5-48d5-bc85-e9d17d82829d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:26.338643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1a3c1-e0a5-48d5-bc85-e9d17d82829d None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:26.339878 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 646/1293] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:21:26.345430 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa471894-559a-45af-b6d6-636bb039bc34 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:26.345865 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa471894-559a-45af-b6d6-636bb039bc34 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:26.345997 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa471894-559a-45af-b6d6-636bb039bc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:26.364425 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-aa471894-559a-45af-b6d6-636bb039bc34 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:21:26.365589 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-aa471894-559a-45af-b6d6-636bb039bc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:21:26.393153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:26.432018 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 648/1294] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:21:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:21:26.453911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1a3c1-e0a5-48d5-bc85-e9d17d82829d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:26.485143 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:26.553670 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1a3c1-e0a5-48d5-bc85-e9d17d82829d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:26.585755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a86b70-713a-4c7e-aea2-ec94423f1a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:26.586349 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a86b70-713a-4c7e-aea2-ec94423f1a85 None admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:26.586625 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a86b70-713a-4c7e-aea2-ec94423f1a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:26.586893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a86b70-713a-4c7e-aea2-ec94423f1a85 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:26.587575 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a86b70-713a-4c7e-aea2-ec94423f1a85 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:26.588997 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a86b70-713a-4c7e-aea2-ec94423f1a85 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:26.602820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 647/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Sat Aug 6 12:21:26 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:21:28.623659 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077c96c-ca0f-42fc-9de1-56e24d64ca84 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:28.624837 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077c96c-ca0f-42fc-9de1-56e24d64ca84 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:28.624837 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077c96c-ca0f-42fc-9de1-56e24d64ca84 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:28.626139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 649/1296] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:21:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:21:28.631276 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304eaa6-2b67-4919-a4aa-cb871cc0dd88 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:28.631700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304eaa6-2b67-4919-a4aa-cb871cc0dd88 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:28.631829 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304eaa6-2b67-4919-a4aa-cb871cc0dd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:28.651413 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-5304eaa6-2b67-4919-a4aa-cb871cc0dd88 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:21:28.652898 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5304eaa6-2b67-4919-a4aa-cb871cc0dd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:21:28.734429 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 648/1297] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:21:28 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:21:28.757262 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077c96c-ca0f-42fc-9de1-56e24d64ca84 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:28.857085 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077c96c-ca0f-42fc-9de1-56e24d64ca84 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:28.890937 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f158b-4206-415d-9f3f-95c0fbc4a96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:28.891502 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f158b-4206-415d-9f3f-95c0fbc4a96d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:28.891779 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f158b-4206-415d-9f3f-95c0fbc4a96d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:28.892050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f158b-4206-415d-9f3f-95c0fbc4a96d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:28.892693 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f158b-4206-415d-9f3f-95c0fbc4a96d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:28.894051 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f158b-4206-415d-9f3f-95c0fbc4a96d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:28.902545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-bc0f158b-4206-415d-9f3f-95c0fbc4a96d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 12:21:28.902545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:28.902545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:21:28.902545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:21:28.902545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:21:28.902545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:21:28.902545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:21:28.902545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:21:28.902545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 12:21:28.902545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:21:28.902545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:21:28.902545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:21:28.902545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:28.902545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:28.902545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:28.902545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:28.902545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:28.903795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:28.903795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:28.903795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:28.903795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:28.903795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:28.903795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:28.903795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:21:28.903795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:21:28.903795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:21:28.903795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:21:28.903795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:28.903795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:28.903795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:28.903795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:28.903795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:28.903795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:28.904814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:28.904814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:28.904814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:28.904814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:28.904814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:28.904814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:28.904814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:28.904814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:28.904814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:28.904814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:28.904814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:21:28.904814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:21:28.904814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 12:21:28.904814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:21:28.904814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 650/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Sat Aug 6 12:21:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:21:28.909648 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304eaa6-2b67-4919-a4aa-cb871cc0dd88 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:28.927734 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:29.006739 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304eaa6-2b67-4919-a4aa-cb871cc0dd88 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:29.038763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c94cb-f629-4bf2-b81e-82912c28515c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:29.039373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c94cb-f629-4bf2-b81e-82912c28515c None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:29.039651 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c94cb-f629-4bf2-b81e-82912c28515c None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:29.039925 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c94cb-f629-4bf2-b81e-82912c28515c None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:29.040622 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c94cb-f629-4bf2-b81e-82912c28515c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:29.040927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c94cb-f629-4bf2-b81e-82912c28515c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:29.042288 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c94cb-f629-4bf2-b81e-82912c28515c None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:29.047789 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 649/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Sat Aug 6 12:21:28 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:29.053642 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f158b-4206-415d-9f3f-95c0fbc4a96d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:29.084899 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:29.155161 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f158b-4206-415d-9f3f-95c0fbc4a96d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:29.187629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6274e-7ea9-403a-8751-4f3f4e700727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:29.188230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6274e-7ea9-403a-8751-4f3f4e700727 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:29.188535 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6274e-7ea9-403a-8751-4f3f4e700727 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:29.188803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6274e-7ea9-403a-8751-4f3f4e700727 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:29.189541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6274e-7ea9-403a-8751-4f3f4e700727 None admin] RBAC: Adding query filter params (interface=public, service_id=0b69e2fe672646289e802562bb1ce49e, region_id=RegionOne) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:29.189839 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6274e-7ea9-403a-8751-4f3f4e700727 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:29.191185 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6274e-7ea9-403a-8751-4f3f4e700727 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:29.195983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 651/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Sat Aug 6 12:21:29 2022] GET /identity/v3/endpoints?service_id=0b69e2fe672646289e802562bb1ce49e&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:29.201774 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c94cb-f629-4bf2-b81e-82912c28515c None admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:29.280097 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:29.299038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c94cb-f629-4bf2-b81e-82912c28515c None admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:29.331062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac136a00-bf3d-4faa-bae4-86fc046c1801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:29.331663 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac136a00-bf3d-4faa-bae4-86fc046c1801 None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:29.331941 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac136a00-bf3d-4faa-bae4-86fc046c1801 None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:29.332213 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac136a00-bf3d-4faa-bae4-86fc046c1801 None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:29.332884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac136a00-bf3d-4faa-bae4-86fc046c1801 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:29.334242 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac136a00-bf3d-4faa-bae4-86fc046c1801 None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:29.340814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 650/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Sat Aug 6 12:21:29 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:21:31.290867 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b5d37-475c-407b-8d43-f592dc95252d None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:31.291399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b5d37-475c-407b-8d43-f592dc95252d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:31.291399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b5d37-475c-407b-8d43-f592dc95252d None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:31.292835 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 652/1302] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:21:31.298285 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c5d94-8eb3-4073-b5ef-957499fba76f None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:31.298697 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c5d94-8eb3-4073-b5ef-957499fba76f None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:31.298827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c5d94-8eb3-4073-b5ef-957499fba76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:31.316980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3f4c5d94-8eb3-4073-b5ef-957499fba76f None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:21:31.318139 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3f4c5d94-8eb3-4073-b5ef-957499fba76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:21:31.381378 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 651/1303] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:21:31 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 06 12:21:31.403557 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b5d37-475c-407b-8d43-f592dc95252d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:31.495790 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:31.502749 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b5d37-475c-407b-8d43-f592dc95252d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:31.535435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723852e7-c4a8-4bf9-9034-186a505f194a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:31.536049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723852e7-c4a8-4bf9-9034-186a505f194a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:31.536348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723852e7-c4a8-4bf9-9034-186a505f194a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:31.536624 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723852e7-c4a8-4bf9-9034-186a505f194a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:31.537240 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723852e7-c4a8-4bf9-9034-186a505f194a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:31.538579 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723852e7-c4a8-4bf9-9034-186a505f194a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:31.547367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-723852e7-c4a8-4bf9-9034-186a505f194a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 12:21:31.547367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:31.547367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 06 12:21:31.547367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 06 12:21:31.547367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 06 12:21:31.547367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 06 12:21:31.547367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 06 12:21:31.547367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:21:31.547367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 12:21:31.547367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:21:31.547367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 06 12:21:31.547367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:21:31.547367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 06 12:21:31.547367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 06 12:21:31.547367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 06 12:21:31.547367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 06 12:21:31.547367 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 06 12:21:31.548529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 06 12:21:31.548529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 06 12:21:31.548529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 06 12:21:31.548529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 06 12:21:31.548529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 06 12:21:31.548529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 06 12:21:31.548529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 06 12:21:31.548529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 06 12:21:31.548529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 06 12:21:31.548529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 06 12:21:31.548529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 06 12:21:31.548529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 06 12:21:31.548529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 06 12:21:31.548529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 06 12:21:31.548529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 06 12:21:31.548529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 06 12:21:31.549374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 06 12:21:31.549374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 06 12:21:31.549374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 06 12:21:31.549374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 06 12:21:31.549374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 06 12:21:31.549374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 06 12:21:31.549374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 06 12:21:31.549374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 06 12:21:31.549374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 06 12:21:31.549374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 06 12:21:31.549374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 06 12:21:31.549374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 06 12:21:31.549374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 06 12:21:31.549374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 06 12:21:31.549374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 653/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Sat Aug 6 12:21:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 06 12:21:31.554304 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c5d94-8eb3-4073-b5ef-957499fba76f None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:31.626543 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:31.649517 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c5d94-8eb3-4073-b5ef-957499fba76f None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:31.681350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ff840-f48f-41fc-ae4f-33c43cf53ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:31.681950 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ff840-f48f-41fc-ae4f-33c43cf53ade None admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:31.682225 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ff840-f48f-41fc-ae4f-33c43cf53ade None admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:31.682493 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ff840-f48f-41fc-ae4f-33c43cf53ade None admin] PATH_INFO: `/v3/services` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:31.683156 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ff840-f48f-41fc-ae4f-33c43cf53ade None admin] RBAC: Adding query filter params (name=placement) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:21:31.683452 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ff840-f48f-41fc-ae4f-33c43cf53ade None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:31.684834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ff840-f48f-41fc-ae4f-33c43cf53ade None admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:31.690251 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 652/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Sat Aug 6 12:21:31 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:21:31.695983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723852e7-c4a8-4bf9-9034-186a505f194a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:21:31.791979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723852e7-c4a8-4bf9-9034-186a505f194a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:21:31.824049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f9db6-9d0f-4459-a2b8-cc727b49415a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:21:31.824615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f9db6-9d0f-4459-a2b8-cc727b49415a None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:21:31.824897 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f9db6-9d0f-4459-a2b8-cc727b49415a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:21:31.825171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f9db6-9d0f-4459-a2b8-cc727b49415a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:21:31.825863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f9db6-9d0f-4459-a2b8-cc727b49415a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:21:31.827268 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f9db6-9d0f-4459-a2b8-cc727b49415a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:21:31.846237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 654/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:21:31 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:21:41.687871 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=30 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:21:41.842137 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:22:34.553585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1d299-8ed1-4a45-bc51-cc7c5cf47a87 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:34.554186 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1d299-8ed1-4a45-bc51-cc7c5cf47a87 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:34.554186 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1d299-8ed1-4a45-bc51-cc7c5cf47a87 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:34.556477 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 653/1307] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:22:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:22:34.561605 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241aba18-567d-4371-a169-4054c1d6c026 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:34.562601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241aba18-567d-4371-a169-4054c1d6c026 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:34.563120 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241aba18-567d-4371-a169-4054c1d6c026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:34.737849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-241aba18-567d-4371-a169-4054c1d6c026 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:22:34.739716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-241aba18-567d-4371-a169-4054c1d6c026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:34.908832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 655/1308] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:22:34.981875 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e48ca-fbd2-4cba-9b87-ade17713f2cb None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:34.982262 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e48ca-fbd2-4cba-9b87-ade17713f2cb None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:34.982453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e48ca-fbd2-4cba-9b87-ade17713f2cb None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:34.983833 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 654/1309] 10.222.0.20 () {54 vars in 965 bytes} [Sat Aug 6 12:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:22:34.994195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd25663-51f2-4c5f-a8b0-925606657a4c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:34.994482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd25663-51f2-4c5f-a8b0-925606657a4c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:34.994760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd25663-51f2-4c5f-a8b0-925606657a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:35.187603 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1cd25663-51f2-4c5f-a8b0-925606657a4c None None] MFA Rules not processed for user `55563aaa662246c5aaf107145475bd76`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:22:35.191714 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1cd25663-51f2-4c5f-a8b0-925606657a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:35.239623 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:22:35.327772 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 656/1310] 10.222.0.20 () {58 vars in 1062 bytes} [Sat Aug 6 12:22:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:22:35.337941 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e48ca-fbd2-4cba-9b87-ade17713f2cb None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:22:35.340643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-377e48ca-fbd2-4cba-9b87-ade17713f2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:35.466134 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e48ca-fbd2-4cba-9b87-ade17713f2cb None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:22:35.543877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d9433-1e76-4131-9500-7bd55797363d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:22:35.544519 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d9433-1e76-4131-9500-7bd55797363d service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:35.544797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d9433-1e76-4131-9500-7bd55797363d service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:35.545066 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d9433-1e76-4131-9500-7bd55797363d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:35.546687 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-448d9433-1e76-4131-9500-7bd55797363d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:35.604050 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:22:35.675277 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d9433-1e76-4131-9500-7bd55797363d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:22:35.676789 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d9433-1e76-4131-9500-7bd55797363d service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:22:35.700550 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 655/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:22:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:22:38.055791 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553f381-4b81-4372-9590-e5a37bdd0036 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:38.056396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553f381-4b81-4372-9590-e5a37bdd0036 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:38.056396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553f381-4b81-4372-9590-e5a37bdd0036 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:38.058647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 657/1312] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:22:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:22:38.064436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9f1fe-ea40-48f0-ba71-548a70cf38f2 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:38.065023 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9f1fe-ea40-48f0-ba71-548a70cf38f2 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:38.065023 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9f1fe-ea40-48f0-ba71-548a70cf38f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:38.141207 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c9b9f1fe-ea40-48f0-ba71-548a70cf38f2 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:22:38.142432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c9b9f1fe-ea40-48f0-ba71-548a70cf38f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:38.241721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 656/1313] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:22:38.279975 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c553f381-4b81-4372-9590-e5a37bdd0036 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:22:38.281604 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c553f381-4b81-4372-9590-e5a37bdd0036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:38.389904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c553f381-4b81-4372-9590-e5a37bdd0036 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:22:38.463011 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:22:38.465985 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad8789-4f1b-4f63-964c-4c2dbdb6c706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:22:38.466607 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad8789-4f1b-4f63-964c-4c2dbdb6c706 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:38.466882 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad8789-4f1b-4f63-964c-4c2dbdb6c706 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:38.467165 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad8789-4f1b-4f63-964c-4c2dbdb6c706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:38.469510 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b4ad8789-4f1b-4f63-964c-4c2dbdb6c706 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:38.578727 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad8789-4f1b-4f63-964c-4c2dbdb6c706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:22:38.580162 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad8789-4f1b-4f63-964c-4c2dbdb6c706 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:22:38.599016 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 658/1314] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:22:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:22:40.893835 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae4cb3-f8e7-4b79-a663-f23e67e6c595 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:40.894386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae4cb3-f8e7-4b79-a663-f23e67e6c595 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:40.894386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae4cb3-f8e7-4b79-a663-f23e67e6c595 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:40.895711 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 657/1315] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:22:40.900742 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1efdb-e381-4638-9afd-0a9e2dfe2e5e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:40.901172 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1efdb-e381-4638-9afd-0a9e2dfe2e5e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:40.901869 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1efdb-e381-4638-9afd-0a9e2dfe2e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:40.965821 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c8c1efdb-e381-4638-9afd-0a9e2dfe2e5e None None] MFA Rules not processed for user `91c44362773b4667825e3087cb73188e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:22:40.967017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c8c1efdb-e381-4638-9afd-0a9e2dfe2e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:41.017743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:22:41.061384 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 659/1316] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:22:40 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:22:41.116760 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a7a62-0cae-44f0-89da-4a724ea82f5a None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:41.117041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a7a62-0cae-44f0-89da-4a724ea82f5a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:41.117316 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a7a62-0cae-44f0-89da-4a724ea82f5a None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:41.118656 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 658/1317] 10.222.0.20 () {54 vars in 965 bytes} [Sat Aug 6 12:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:22:41.124752 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca1a9b-a167-4192-9002-699732055365 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:41.125041 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca1a9b-a167-4192-9002-699732055365 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:41.125315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca1a9b-a167-4192-9002-699732055365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:41.199693 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f7ca1a9b-a167-4192-9002-699732055365 None None] MFA Rules not processed for user `55563aaa662246c5aaf107145475bd76`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:22:41.200842 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f7ca1a9b-a167-4192-9002-699732055365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:41.273925 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 660/1318] 10.222.0.20 () {58 vars in 1062 bytes} [Sat Aug 6 12:22:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:22:41.283376 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a7a62-0cae-44f0-89da-4a724ea82f5a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:22:41.284910 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d18a7a62-0cae-44f0-89da-4a724ea82f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:41.294935 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:22:41.387375 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a7a62-0cae-44f0-89da-4a724ea82f5a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:22:41.456797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8441942-f9f4-4eae-af73-6448b66854e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:22:41.457330 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8441942-f9f4-4eae-af73-6448b66854e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:41.457603 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8441942-f9f4-4eae-af73-6448b66854e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:41.457870 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8441942-f9f4-4eae-af73-6448b66854e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:41.459460 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d8441942-f9f4-4eae-af73-6448b66854e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:41.521298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:22:41.567543 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8441942-f9f4-4eae-af73-6448b66854e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:22:41.569012 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8441942-f9f4-4eae-af73-6448b66854e0 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:22:41.586768 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 659/1319] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:22:41 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:22:44.532457 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f71ff4-f8f9-4146-ba94-457d0e68527b None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:44.532968 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f71ff4-f8f9-4146-ba94-457d0e68527b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:44.533061 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f71ff4-f8f9-4146-ba94-457d0e68527b None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:44.534380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 661/1320] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:22:44.539593 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95877ba-ceeb-4709-ae42-e531d0447958 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:44.540026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95877ba-ceeb-4709-ae42-e531d0447958 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:44.540153 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95877ba-ceeb-4709-ae42-e531d0447958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:44.604820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-e95877ba-ceeb-4709-ae42-e531d0447958 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:22:44.605990 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e95877ba-ceeb-4709-ae42-e531d0447958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:44.702833 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 660/1321] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:22:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:22:44.741269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f71ff4-f8f9-4146-ba94-457d0e68527b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:22:44.742877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-89f71ff4-f8f9-4146-ba94-457d0e68527b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:44.792280 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:22:44.844706 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f71ff4-f8f9-4146-ba94-457d0e68527b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:22:44.916019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa88e1d-4385-49ac-9d99-2a02dae51ffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:22:44.917186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa88e1d-4385-49ac-9d99-2a02dae51ffc service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:44.917460 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa88e1d-4385-49ac-9d99-2a02dae51ffc service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:44.917729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa88e1d-4385-49ac-9d99-2a02dae51ffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:44.919324 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1aa88e1d-4385-49ac-9d99-2a02dae51ffc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:45.012438 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:22:45.026355 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa88e1d-4385-49ac-9d99-2a02dae51ffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:22:45.027779 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa88e1d-4385-49ac-9d99-2a02dae51ffc service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:22:45.045766 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 662/1322] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:22:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:22:47.735744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082c4c2-a61e-4e4b-8221-e5cd4703f3d1 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:47.736278 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082c4c2-a61e-4e4b-8221-e5cd4703f3d1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:47.736278 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082c4c2-a61e-4e4b-8221-e5cd4703f3d1 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:47.738057 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 661/1323] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:22:47.743287 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cee35e-b5cf-4bb9-9dd0-e917f95ae21a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:47.743724 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cee35e-b5cf-4bb9-9dd0-e917f95ae21a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:47.743830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cee35e-b5cf-4bb9-9dd0-e917f95ae21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:47.809615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-86cee35e-b5cf-4bb9-9dd0-e917f95ae21a None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:22:47.810805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-86cee35e-b5cf-4bb9-9dd0-e917f95ae21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:47.914664 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 663/1324] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:22:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:22:47.951869 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082c4c2-a61e-4e4b-8221-e5cd4703f3d1 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:22:47.953437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a082c4c2-a61e-4e4b-8221-e5cd4703f3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:47.974485 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:22:48.064434 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082c4c2-a61e-4e4b-8221-e5cd4703f3d1 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:22:48.136022 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c894733-9e67-47df-9377-13ee042d443f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:22:48.136780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c894733-9e67-47df-9377-13ee042d443f service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:48.137087 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c894733-9e67-47df-9377-13ee042d443f service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:48.137360 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c894733-9e67-47df-9377-13ee042d443f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:48.138971 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7c894733-9e67-47df-9377-13ee042d443f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:48.209220 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:22:48.247136 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c894733-9e67-47df-9377-13ee042d443f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:22:48.248652 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c894733-9e67-47df-9377-13ee042d443f service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:22:48.266567 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 662/1325] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:22:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:22:50.300648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ca54b-5968-4203-b9c9-ecc35f66e085 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:50.301162 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ca54b-5968-4203-b9c9-ecc35f66e085 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:50.301275 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ca54b-5968-4203-b9c9-ecc35f66e085 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:50.302547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 664/1326] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:22:50.307633 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206c401-8fcb-4b36-a180-be67c6382ebf None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:50.308082 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206c401-8fcb-4b36-a180-be67c6382ebf None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:50.308370 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206c401-8fcb-4b36-a180-be67c6382ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:50.373771 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c206c401-8fcb-4b36-a180-be67c6382ebf None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:22:50.374971 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c206c401-8fcb-4b36-a180-be67c6382ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:50.470957 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 663/1327] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:22:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:22:50.506503 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ca54b-5968-4203-b9c9-ecc35f66e085 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:22:50.508016 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1f5ca54b-5968-4203-b9c9-ecc35f66e085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:50.567947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:22:50.612353 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ca54b-5968-4203-b9c9-ecc35f66e085 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:22:50.687398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340e063-1736-48d0-a1f1-a4a3cea779d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:22:50.688019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340e063-1736-48d0-a1f1-a4a3cea779d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:50.688313 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340e063-1736-48d0-a1f1-a4a3cea779d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:50.688596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340e063-1736-48d0-a1f1-a4a3cea779d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:50.690203 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6340e063-1736-48d0-a1f1-a4a3cea779d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:50.801709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:22:50.805058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340e063-1736-48d0-a1f1-a4a3cea779d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:22:50.806873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340e063-1736-48d0-a1f1-a4a3cea779d5 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:22:50.825706 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 665/1328] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:22:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:22:52.871107 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca855c57-0f11-41ce-b548-fd3e318cbf50 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:52.871628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca855c57-0f11-41ce-b548-fd3e318cbf50 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:52.871628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca855c57-0f11-41ce-b548-fd3e318cbf50 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:52.873016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 664/1329] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:22:52.878024 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0df83f-7ee8-43d4-910e-4959dcbd4a6f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:52.878438 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0df83f-7ee8-43d4-910e-4959dcbd4a6f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:52.878570 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0df83f-7ee8-43d4-910e-4959dcbd4a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:52.943818 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4e0df83f-7ee8-43d4-910e-4959dcbd4a6f None None] MFA Rules not processed for user `91c44362773b4667825e3087cb73188e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:22:52.945042 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4e0df83f-7ee8-43d4-910e-4959dcbd4a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:53.042653 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 666/1330] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:22:52 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:22:53.080239 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca855c57-0f11-41ce-b548-fd3e318cbf50 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:22:53.081787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ca855c57-0f11-41ce-b548-fd3e318cbf50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:53.113403 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:22:53.190402 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca855c57-0f11-41ce-b548-fd3e318cbf50 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:22:53.262446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d9337-af3c-43c9-8f85-a9c471b1b384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:22:53.263059 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d9337-af3c-43c9-8f85-a9c471b1b384 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:53.263335 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d9337-af3c-43c9-8f85-a9c471b1b384 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:53.263606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d9337-af3c-43c9-8f85-a9c471b1b384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:53.265250 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-286d9337-af3c-43c9-8f85-a9c471b1b384 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:53.342524 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:22:53.375361 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d9337-af3c-43c9-8f85-a9c471b1b384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:22:53.376841 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d9337-af3c-43c9-8f85-a9c471b1b384 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:22:53.394873 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 665/1331] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:22:53 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:22:55.694174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eced3d2-2b65-466e-a6e7-15c467d1fa76 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:55.694678 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eced3d2-2b65-466e-a6e7-15c467d1fa76 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:55.694773 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eced3d2-2b65-466e-a6e7-15c467d1fa76 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:55.696403 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 667/1332] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:22:55.701459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633f174-3afd-438c-9e91-39b98fae6ea9 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:55.701851 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633f174-3afd-438c-9e91-39b98fae6ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:55.702017 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633f174-3afd-438c-9e91-39b98fae6ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:55.767114 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-4633f174-3afd-438c-9e91-39b98fae6ea9 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:22:55.768326 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4633f174-3afd-438c-9e91-39b98fae6ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:55.863275 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 666/1333] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:22:55.899477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eced3d2-2b65-466e-a6e7-15c467d1fa76 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:22:55.901741 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1eced3d2-2b65-466e-a6e7-15c467d1fa76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:55.969926 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:22:56.004768 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eced3d2-2b65-466e-a6e7-15c467d1fa76 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:22:56.077449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70bd29a-29e2-4ca8-ab65-ee06d9531b3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:22:56.078059 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bd29a-29e2-4ca8-ab65-ee06d9531b3a service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:56.078339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bd29a-29e2-4ca8-ab65-ee06d9531b3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:56.078608 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bd29a-29e2-4ca8-ab65-ee06d9531b3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:56.080222 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c70bd29a-29e2-4ca8-ab65-ee06d9531b3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:56.190839 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70bd29a-29e2-4ca8-ab65-ee06d9531b3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:22:56.192227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70bd29a-29e2-4ca8-ab65-ee06d9531b3a service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:22:56.202042 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:22:56.210216 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 668/1334] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:22:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:22:58.257526 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d228d-096f-4d21-9a0c-755ef475e37b None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:58.258045 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d228d-096f-4d21-9a0c-755ef475e37b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:58.258045 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d228d-096f-4d21-9a0c-755ef475e37b None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:58.259319 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 667/1335] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:22:58.265100 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5d9d3-6ef2-4289-9b3f-9e10251f3e8e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:58.265501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5d9d3-6ef2-4289-9b3f-9e10251f3e8e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:58.265646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5d9d3-6ef2-4289-9b3f-9e10251f3e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:58.330798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0da5d9d3-6ef2-4289-9b3f-9e10251f3e8e None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:22:58.331997 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0da5d9d3-6ef2-4289-9b3f-9e10251f3e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:58.424535 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 669/1336] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:22:58.464827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d228d-096f-4d21-9a0c-755ef475e37b None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:22:58.466351 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-cd4d228d-096f-4d21-9a0c-755ef475e37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:58.505901 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:22:58.569178 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d228d-096f-4d21-9a0c-755ef475e37b None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:22:58.647729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b2b59-e388-4307-b20c-4909a448f4e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:22:58.648367 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b2b59-e388-4307-b20c-4909a448f4e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:22:58.648651 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b2b59-e388-4307-b20c-4909a448f4e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:22:58.648924 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b2b59-e388-4307-b20c-4909a448f4e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:22:58.650538 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7a6b2b59-e388-4307-b20c-4909a448f4e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:22:58.731737 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:22:58.757473 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b2b59-e388-4307-b20c-4909a448f4e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:22:58.758909 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b2b59-e388-4307-b20c-4909a448f4e7 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:22:58.778432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 668/1337] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:22:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:23:01.237815 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda345fb-a2e2-4155-82b6-ff37be31781c None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:01.238419 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda345fb-a2e2-4155-82b6-ff37be31781c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:01.238419 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda345fb-a2e2-4155-82b6-ff37be31781c None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:01.239720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 670/1338] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:01.245349 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af6f8a-17b8-4fc8-a819-7c7cac470216 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:01.245777 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af6f8a-17b8-4fc8-a819-7c7cac470216 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:01.245879 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af6f8a-17b8-4fc8-a819-7c7cac470216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:01.314081 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-d9af6f8a-17b8-4fc8-a819-7c7cac470216 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:01.315287 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d9af6f8a-17b8-4fc8-a819-7c7cac470216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:01.407593 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 669/1339] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:23:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:23:01.444201 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda345fb-a2e2-4155-82b6-ff37be31781c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:23:01.445734 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bda345fb-a2e2-4155-82b6-ff37be31781c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:01.526532 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:01.554654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda345fb-a2e2-4155-82b6-ff37be31781c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:23:01.628637 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f581b-d314-4d8e-94ed-034f537e5f67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:23:01.629176 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f581b-d314-4d8e-94ed-034f537e5f67 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:01.629454 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f581b-d314-4d8e-94ed-034f537e5f67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:01.629725 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f581b-d314-4d8e-94ed-034f537e5f67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:01.631334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8c1f581b-d314-4d8e-94ed-034f537e5f67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:01.740986 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1f581b-d314-4d8e-94ed-034f537e5f67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:23:01.742690 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1f581b-d314-4d8e-94ed-034f537e5f67 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:23:01.760858 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 671/1340] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:23:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:23:05.764525 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf99eb3-1d41-4ccb-a7b0-7e0342202f56 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:05.765088 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf99eb3-1d41-4ccb-a7b0-7e0342202f56 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:05.765088 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf99eb3-1d41-4ccb-a7b0-7e0342202f56 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:05.766429 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 670/1341] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:05.771517 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71462c-2701-48d8-839a-20ec09af933e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:05.771911 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71462c-2701-48d8-839a-20ec09af933e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:05.772182 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71462c-2701-48d8-839a-20ec09af933e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:05.778464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:05.840739 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0e71462c-2701-48d8-839a-20ec09af933e None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:05.841861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0e71462c-2701-48d8-839a-20ec09af933e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:05.940721 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 672/1342] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:23:05.978900 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf99eb3-1d41-4ccb-a7b0-7e0342202f56 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:23:05.980453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2bf99eb3-1d41-4ccb-a7b0-7e0342202f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:06.033534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:06.088659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf99eb3-1d41-4ccb-a7b0-7e0342202f56 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:23:06.162916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ad58b-a12b-4266-9e73-773994106968 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:23:06.163444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ad58b-a12b-4266-9e73-773994106968 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:06.163718 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ad58b-a12b-4266-9e73-773994106968 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:06.163988 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ad58b-a12b-4266-9e73-773994106968 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:06.166225 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0f8ad58b-a12b-4266-9e73-773994106968 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:06.256381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:06.272837 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ad58b-a12b-4266-9e73-773994106968 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:23:06.274275 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ad58b-a12b-4266-9e73-773994106968 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:23:06.293273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 671/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:23:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:23:08.905448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5560167-06bb-4aa0-b2cc-e904c480206b None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:08.906025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5560167-06bb-4aa0-b2cc-e904c480206b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:08.906025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5560167-06bb-4aa0-b2cc-e904c480206b None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:08.907411 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 673/1344] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:08.913069 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babb861-5ed0-4bae-b432-d225b5b91381 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:08.913558 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babb861-5ed0-4bae-b432-d225b5b91381 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:08.913681 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babb861-5ed0-4bae-b432-d225b5b91381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:08.985744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-4babb861-5ed0-4bae-b432-d225b5b91381 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:08.987232 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4babb861-5ed0-4bae-b432-d225b5b91381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:09.091502 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 672/1345] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:23:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:23:09.129459 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5560167-06bb-4aa0-b2cc-e904c480206b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:23:09.130969 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f5560167-06bb-4aa0-b2cc-e904c480206b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:09.217601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:09.235551 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5560167-06bb-4aa0-b2cc-e904c480206b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:23:09.311310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1299d1f6-0489-41ef-a722-b0669c304a24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:23:09.311919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299d1f6-0489-41ef-a722-b0669c304a24 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:09.312196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299d1f6-0489-41ef-a722-b0669c304a24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:09.312501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299d1f6-0489-41ef-a722-b0669c304a24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:09.314096 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1299d1f6-0489-41ef-a722-b0669c304a24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:09.420887 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1299d1f6-0489-41ef-a722-b0669c304a24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:23:09.422311 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1299d1f6-0489-41ef-a722-b0669c304a24 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:23:09.440107 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 674/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:23:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:23:13.453051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4f898-bf6d-4ea3-bb80-0cdc9cba9b10 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:13.453612 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4f898-bf6d-4ea3-bb80-0cdc9cba9b10 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:13.453612 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4f898-bf6d-4ea3-bb80-0cdc9cba9b10 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:13.454942 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 673/1347] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:13.460056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a72ed-c628-45b1-a408-e604d4c85deb None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:13.460532 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a72ed-c628-45b1-a408-e604d4c85deb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:13.460751 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a72ed-c628-45b1-a408-e604d4c85deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:13.477255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:13.530697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-433a72ed-c628-45b1-a408-e604d4c85deb None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:13.531912 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-433a72ed-c628-45b1-a408-e604d4c85deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:13.642335 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 675/1348] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:23:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:23:13.680191 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4f898-bf6d-4ea3-bb80-0cdc9cba9b10 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:23:13.682263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a5f4f898-bf6d-4ea3-bb80-0cdc9cba9b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:13.741969 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:13.792616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4f898-bf6d-4ea3-bb80-0cdc9cba9b10 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:23:13.869840 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814482ae-21b7-4be3-8380-1e3a60f68b84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:23:13.870412 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814482ae-21b7-4be3-8380-1e3a60f68b84 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:13.870716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814482ae-21b7-4be3-8380-1e3a60f68b84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:13.870984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814482ae-21b7-4be3-8380-1e3a60f68b84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:13.872652 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-814482ae-21b7-4be3-8380-1e3a60f68b84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:13.983803 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:13.988128 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814482ae-21b7-4be3-8380-1e3a60f68b84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:23:13.989851 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814482ae-21b7-4be3-8380-1e3a60f68b84 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:23:14.010998 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 674/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:23:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:23:18.916091 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f8af3-4895-4ee4-a02f-264d8400aa07 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:18.916650 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f8af3-4895-4ee4-a02f-264d8400aa07 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:18.917119 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f8af3-4895-4ee4-a02f-264d8400aa07 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:18.919139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 676/1350] 10.222.0.20 () {54 vars in 921 bytes} [Sat Aug 6 12:23:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:18.925318 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058180a5-1c3b-4636-b9ec-26ea84987e6b None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:18.928141 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058180a5-1c3b-4636-b9ec-26ea84987e6b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:18.928141 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058180a5-1c3b-4636-b9ec-26ea84987e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:19.016526 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-058180a5-1c3b-4636-b9ec-26ea84987e6b None None] MFA Rules not processed for user `e2244655ba304b9c980b5e68772eefc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:19.016526 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-058180a5-1c3b-4636-b9ec-26ea84987e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:19.161024 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 675/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Aug 6 12:23:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:23:23.337863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0311bca-bfc9-4d7a-a08b-ac21649944af None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:23.338396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0311bca-bfc9-4d7a-a08b-ac21649944af None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:23.338396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0311bca-bfc9-4d7a-a08b-ac21649944af None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:23.339745 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 677/1352] 10.222.0.20 () {54 vars in 921 bytes} [Sat Aug 6 12:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:23.345830 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef43dda-3ea9-404d-ac46-a43ab064e59e None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:23.346223 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef43dda-3ea9-404d-ac46-a43ab064e59e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:23.346468 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef43dda-3ea9-404d-ac46-a43ab064e59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:23.401500 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:23.426020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-9ef43dda-3ea9-404d-ac46-a43ab064e59e None None] MFA Rules not processed for user `e2244655ba304b9c980b5e68772eefc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:23.427215 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9ef43dda-3ea9-404d-ac46-a43ab064e59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:23.536337 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 676/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Aug 6 12:23:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:23:23.640542 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=59 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:24.865356 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9ebfc-d6f7-4017-8e20-3d4e15ef44f6 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:24.865987 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9ebfc-d6f7-4017-8e20-3d4e15ef44f6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:24.865987 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9ebfc-d6f7-4017-8e20-3d4e15ef44f6 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:24.867777 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 678/1354] 10.222.0.20 () {54 vars in 921 bytes} [Sat Aug 6 12:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:24.872884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8aaea-eaeb-4f1f-b192-8b4bd7d8656a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:24.873270 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8aaea-eaeb-4f1f-b192-8b4bd7d8656a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:24.873582 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8aaea-eaeb-4f1f-b192-8b4bd7d8656a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:24.949425 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3ba8aaea-eaeb-4f1f-b192-8b4bd7d8656a None None] MFA Rules not processed for user `e2244655ba304b9c980b5e68772eefc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:24.950612 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3ba8aaea-eaeb-4f1f-b192-8b4bd7d8656a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:24.996756 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:25.063912 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 677/1355] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Aug 6 12:23:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:23:28.512855 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42d36b-26f4-47d7-ba81-a2e372365ee9 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:28.513701 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42d36b-26f4-47d7-ba81-a2e372365ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:28.514146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42d36b-26f4-47d7-ba81-a2e372365ee9 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:28.519597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 679/1356] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:23:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:28.521335 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918fc6e-9e1e-4481-ad4e-aa42ea3a3693 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:28.522574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918fc6e-9e1e-4481-ad4e-aa42ea3a3693 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:28.522943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918fc6e-9e1e-4481-ad4e-aa42ea3a3693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:28.605425 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-4918fc6e-9e1e-4481-ad4e-aa42ea3a3693 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:28.607080 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4918fc6e-9e1e-4481-ad4e-aa42ea3a3693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:28.715451 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 678/1357] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:23:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:23:28.769341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b5e03-138a-4109-93a8-5a9a11383f60 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:28.769881 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b5e03-138a-4109-93a8-5a9a11383f60 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:28.770446 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b5e03-138a-4109-93a8-5a9a11383f60 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:28.772288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 680/1358] 10.222.0.20 () {54 vars in 962 bytes} [Sat Aug 6 12:23:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:28.780275 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b47a0b-021c-4c24-bd6f-cc21e5ce2cae None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:28.780587 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b47a0b-021c-4c24-bd6f-cc21e5ce2cae None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:28.780854 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b47a0b-021c-4c24-bd6f-cc21e5ce2cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:28.798439 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:28.865666 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-23b47a0b-021c-4c24-bd6f-cc21e5ce2cae None None] MFA Rules not processed for user `524b728bf63c4ca88e0f25e531cf5056`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:28.866929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-23b47a0b-021c-4c24-bd6f-cc21e5ce2cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:28.986533 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 679/1359] 10.222.0.20 () {58 vars in 1059 bytes} [Sat Aug 6 12:23:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:23:28.997442 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b5e03-138a-4109-93a8-5a9a11383f60 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:23:29.002317 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e91b5e03-138a-4109-93a8-5a9a11383f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:29.160577 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b5e03-138a-4109-93a8-5a9a11383f60 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:23:29.238003 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:29.280804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacd974-18e4-485d-b4c7-f5bfd29cc2d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:23:29.281455 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacd974-18e4-485d-b4c7-f5bfd29cc2d7 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:29.281703 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacd974-18e4-485d-b4c7-f5bfd29cc2d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:29.281976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacd974-18e4-485d-b4c7-f5bfd29cc2d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:29.283582 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2aacd974-18e4-485d-b4c7-f5bfd29cc2d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:29.379462 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aacd974-18e4-485d-b4c7-f5bfd29cc2d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:23:29.381146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aacd974-18e4-485d-b4c7-f5bfd29cc2d7 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:23:29.400740 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 681/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:23:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:23:31.680422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a670e-2eb1-4be7-a522-1c8521ee4b4b None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:31.680959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a670e-2eb1-4be7-a522-1c8521ee4b4b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:31.680959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a670e-2eb1-4be7-a522-1c8521ee4b4b None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:31.682382 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 680/1361] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:31.687418 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7aea9-2521-4b0c-989e-1d51957aec62 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:31.687868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7aea9-2521-4b0c-989e-1d51957aec62 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:31.687958 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7aea9-2521-4b0c-989e-1d51957aec62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:31.759462 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-65d7aea9-2521-4b0c-989e-1d51957aec62 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:31.760965 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-65d7aea9-2521-4b0c-989e-1d51957aec62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:31.800365 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:31.878989 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 682/1362] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:23:31.922766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a670e-2eb1-4be7-a522-1c8521ee4b4b None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:23:31.924339 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-612a670e-2eb1-4be7-a522-1c8521ee4b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:32.011686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:32.128491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a670e-2eb1-4be7-a522-1c8521ee4b4b None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:23:32.251746 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46634a-424b-45e2-a6a4-ea73c255baed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:23:32.252389 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46634a-424b-45e2-a6a4-ea73c255baed service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:32.252673 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46634a-424b-45e2-a6a4-ea73c255baed service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:32.252948 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46634a-424b-45e2-a6a4-ea73c255baed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:32.254590 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-eb46634a-424b-45e2-a6a4-ea73c255baed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:32.282145 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:32.377429 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb46634a-424b-45e2-a6a4-ea73c255baed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:23:32.379238 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb46634a-424b-45e2-a6a4-ea73c255baed service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:23:32.399109 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 681/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:23:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:23:34.734666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40bdbc-026f-45bc-ba25-248364a93fe7 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:34.735193 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40bdbc-026f-45bc-ba25-248364a93fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:34.735193 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40bdbc-026f-45bc-ba25-248364a93fe7 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:34.736913 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 683/1364] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:34.743263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadad42c-edeb-4b9e-9939-06a7cbcb4378 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:34.744120 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadad42c-edeb-4b9e-9939-06a7cbcb4378 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:34.744510 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadad42c-edeb-4b9e-9939-06a7cbcb4378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:34.823416 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-aadad42c-edeb-4b9e-9939-06a7cbcb4378 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:34.824636 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-aadad42c-edeb-4b9e-9939-06a7cbcb4378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:34.927127 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:34.930444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 682/1365] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:23:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:23:34.974612 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d3174-f160-4165-b016-e2bcb1b654b2 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:34.974988 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d3174-f160-4165-b016-e2bcb1b654b2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:34.975329 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d3174-f160-4165-b016-e2bcb1b654b2 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:34.977445 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 684/1366] 10.222.0.20 () {54 vars in 962 bytes} [Sat Aug 6 12:23:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:34.982712 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a1e7d-7a31-48e5-b3f8-c937dd75f1c8 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:34.983000 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a1e7d-7a31-48e5-b3f8-c937dd75f1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:34.983277 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a1e7d-7a31-48e5-b3f8-c937dd75f1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:35.068906 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-b83a1e7d-7a31-48e5-b3f8-c937dd75f1c8 None None] MFA Rules not processed for user `524b728bf63c4ca88e0f25e531cf5056`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:35.070129 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b83a1e7d-7a31-48e5-b3f8-c937dd75f1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:35.208164 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 683/1367] 10.222.0.20 () {58 vars in 1059 bytes} [Sat Aug 6 12:23:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:23:35.218290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d3174-f160-4165-b016-e2bcb1b654b2 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:23:35.219913 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-188d3174-f160-4165-b016-e2bcb1b654b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:35.383093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d3174-f160-4165-b016-e2bcb1b654b2 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:23:35.403706 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:35.503635 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2aff9-0de5-4e04-96e9-f78faf36913b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:23:35.504254 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2aff9-0de5-4e04-96e9-f78faf36913b service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:35.504554 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2aff9-0de5-4e04-96e9-f78faf36913b service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:35.504824 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2aff9-0de5-4e04-96e9-f78faf36913b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:35.506423 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-16e2aff9-0de5-4e04-96e9-f78faf36913b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:35.598283 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e2aff9-0de5-4e04-96e9-f78faf36913b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:23:35.600030 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e2aff9-0de5-4e04-96e9-f78faf36913b service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:23:35.619538 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 685/1368] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:23:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:23:36.611184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c20ffa-0f62-4280-a9db-8982f9e2cfde None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:36.611700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c20ffa-0f62-4280-a9db-8982f9e2cfde None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:36.611780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c20ffa-0f62-4280-a9db-8982f9e2cfde None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:36.613174 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 684/1369] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:36.619038 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd2a67-fa47-432b-a798-1301ef2a9c98 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:36.619455 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd2a67-fa47-432b-a798-1301ef2a9c98 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:36.619629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd2a67-fa47-432b-a798-1301ef2a9c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:36.652116 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:36.692430 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-31fd2a67-fa47-432b-a798-1301ef2a9c98 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:36.693643 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-31fd2a67-fa47-432b-a798-1301ef2a9c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:36.806624 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 686/1370] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:23:37.178340 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4458dd-e962-4f0b-b4e6-1ba785d0f2ae None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:37.178627 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4458dd-e962-4f0b-b4e6-1ba785d0f2ae None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:37.178901 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4458dd-e962-4f0b-b4e6-1ba785d0f2ae None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:37.180434 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 685/1371] 10.222.0.20 () {54 vars in 964 bytes} [Sat Aug 6 12:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:37.185402 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e8b5d-1b21-4928-9768-9007613ca466 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:37.185694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e8b5d-1b21-4928-9768-9007613ca466 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:37.185974 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e8b5d-1b21-4928-9768-9007613ca466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:37.266661 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-431e8b5d-1b21-4928-9768-9007613ca466 None None] MFA Rules not processed for user `d7a3cbb9c6a34fe2b280873771532e4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:37.267788 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-431e8b5d-1b21-4928-9768-9007613ca466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:37.317226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:37.350129 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 687/1372] 10.222.0.20 () {58 vars in 1061 bytes} [Sat Aug 6 12:23:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:23:37.357555 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4458dd-e962-4f0b-b4e6-1ba785d0f2ae None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:23:37.359072 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2b4458dd-e962-4f0b-b4e6-1ba785d0f2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:37.412613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:37.468036 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4458dd-e962-4f0b-b4e6-1ba785d0f2ae None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:23:37.540391 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c11cb4-f59f-4d6c-952b-b1fd21c54b37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:23:37.541010 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c11cb4-f59f-4d6c-952b-b1fd21c54b37 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:37.541289 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c11cb4-f59f-4d6c-952b-b1fd21c54b37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:37.541561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c11cb4-f59f-4d6c-952b-b1fd21c54b37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:37.543170 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-39c11cb4-f59f-4d6c-952b-b1fd21c54b37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:37.643375 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:37.659714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c11cb4-f59f-4d6c-952b-b1fd21c54b37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:23:37.661023 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c11cb4-f59f-4d6c-952b-b1fd21c54b37 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:23:37.681141 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 686/1373] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:23:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:23:37.721746 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88222e2-c2d3-4d1b-b0c4-d99d6770e41e None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:37.722159 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88222e2-c2d3-4d1b-b0c4-d99d6770e41e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:37.722333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88222e2-c2d3-4d1b-b0c4-d99d6770e41e None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:37.727585 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 688/1374] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:23:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:37.728911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57779f08-f6d6-497f-beea-430b06a4525e None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:37.729294 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57779f08-f6d6-497f-beea-430b06a4525e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:37.729469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57779f08-f6d6-497f-beea-430b06a4525e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:37.800365 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-57779f08-f6d6-497f-beea-430b06a4525e None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:37.801564 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-57779f08-f6d6-497f-beea-430b06a4525e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:37.912203 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 687/1375] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:23:37.955947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88222e2-c2d3-4d1b-b0c4-d99d6770e41e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:23:37.957439 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b88222e2-c2d3-4d1b-b0c4-d99d6770e41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:38.119668 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88222e2-c2d3-4d1b-b0c4-d99d6770e41e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:23:38.172836 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:38.234939 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e621b-cacf-4b54-90fc-4fd6120ea0c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:23:38.235865 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e621b-cacf-4b54-90fc-4fd6120ea0c8 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:38.236141 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e621b-cacf-4b54-90fc-4fd6120ea0c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:38.236441 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e621b-cacf-4b54-90fc-4fd6120ea0c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:38.238076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cb4e621b-cacf-4b54-90fc-4fd6120ea0c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:38.326642 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e621b-cacf-4b54-90fc-4fd6120ea0c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:23:38.328167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e621b-cacf-4b54-90fc-4fd6120ea0c8 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:23:38.346788 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 689/1376] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:23:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:23:40.383469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc4f32-f5a0-4a35-a95f-93bdbcb4db54 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:40.383963 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc4f32-f5a0-4a35-a95f-93bdbcb4db54 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:40.383963 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc4f32-f5a0-4a35-a95f-93bdbcb4db54 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:40.385341 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 688/1377] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:23:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:40.390575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90be17-546f-4792-9dfa-02bd24acbb52 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:40.390978 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90be17-546f-4792-9dfa-02bd24acbb52 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:40.391251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90be17-546f-4792-9dfa-02bd24acbb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:40.466420 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8e90be17-546f-4792-9dfa-02bd24acbb52 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:40.467659 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8e90be17-546f-4792-9dfa-02bd24acbb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:40.474661 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:40.567601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 690/1378] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:23:40.611965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc4f32-f5a0-4a35-a95f-93bdbcb4db54 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:23:40.613599 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9cfc4f32-f5a0-4a35-a95f-93bdbcb4db54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:40.686083 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:40.774788 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc4f32-f5a0-4a35-a95f-93bdbcb4db54 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:23:40.885453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1dbc81-8db3-4463-a90c-c9dac6ff992f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:23:40.886053 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1dbc81-8db3-4463-a90c-c9dac6ff992f service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:40.886327 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1dbc81-8db3-4463-a90c-c9dac6ff992f service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:40.886593 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1dbc81-8db3-4463-a90c-c9dac6ff992f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:40.888186 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0f1dbc81-8db3-4463-a90c-c9dac6ff992f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:40.906542 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:40.978675 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1dbc81-8db3-4463-a90c-c9dac6ff992f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:23:40.980214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1dbc81-8db3-4463-a90c-c9dac6ff992f service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:23:41.002413 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 689/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:23:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:23:43.064406 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8f693-292f-4e29-b631-4971675b4b6e None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:43.064920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8f693-292f-4e29-b631-4971675b4b6e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:43.064920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8f693-292f-4e29-b631-4971675b4b6e None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:43.066376 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 691/1380] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:43.071635 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f99908-2c6f-4204-a934-96056cb28fd8 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:43.072035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f99908-2c6f-4204-a934-96056cb28fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:43.072219 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f99908-2c6f-4204-a934-96056cb28fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:43.142067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-a5f99908-2c6f-4204-a934-96056cb28fd8 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:43.143248 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a5f99908-2c6f-4204-a934-96056cb28fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:43.246228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:43.255954 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 690/1381] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:23:43.298107 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8f693-292f-4e29-b631-4971675b4b6e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:23:43.299618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fcb8f693-292f-4e29-b631-4971675b4b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:43.439792 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:43.451782 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8f693-292f-4e29-b631-4971675b4b6e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:23:43.573782 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66cc96-07c0-4a88-b99d-c0d41438df4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:23:43.574398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66cc96-07c0-4a88-b99d-c0d41438df4b service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:43.574674 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66cc96-07c0-4a88-b99d-c0d41438df4b service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:43.574943 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66cc96-07c0-4a88-b99d-c0d41438df4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:43.576582 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3c66cc96-07c0-4a88-b99d-c0d41438df4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:43.675034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66cc96-07c0-4a88-b99d-c0d41438df4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:23:43.676752 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66cc96-07c0-4a88-b99d-c0d41438df4b service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:23:43.688494 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:43.698405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 692/1382] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:23:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:23:45.920588 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c194af-a418-4ec9-b27a-d12ab989ab53 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:45.921159 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c194af-a418-4ec9-b27a-d12ab989ab53 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:45.921299 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c194af-a418-4ec9-b27a-d12ab989ab53 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:45.922662 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 691/1383] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:45.928358 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efdb77-4533-4341-a130-b07b698c72f7 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:45.928764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efdb77-4533-4341-a130-b07b698c72f7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:45.928922 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efdb77-4533-4341-a130-b07b698c72f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:46.000031 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e5efdb77-4533-4341-a130-b07b698c72f7 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:46.001729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e5efdb77-4533-4341-a130-b07b698c72f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:46.109801 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 693/1384] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:23:46.162974 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c194af-a418-4ec9-b27a-d12ab989ab53 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:23:46.164522 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b5c194af-a418-4ec9-b27a-d12ab989ab53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:46.322037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c194af-a418-4ec9-b27a-d12ab989ab53 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:23:46.391522 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:46.437939 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ef96c-821b-4c26-8160-73452dbdacd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:23:46.438660 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ef96c-821b-4c26-8160-73452dbdacd9 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:46.438940 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ef96c-821b-4c26-8160-73452dbdacd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:46.439209 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ef96c-821b-4c26-8160-73452dbdacd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:46.440868 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8c1ef96c-821b-4c26-8160-73452dbdacd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:46.535474 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ef96c-821b-4c26-8160-73452dbdacd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:23:46.537221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ef96c-821b-4c26-8160-73452dbdacd9 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:23:46.558676 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 692/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:23:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:23:49.128528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f5718-cbd7-46c4-b4c5-50e2bd81092e None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:49.129082 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f5718-cbd7-46c4-b4c5-50e2bd81092e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:49.129286 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f5718-cbd7-46c4-b4c5-50e2bd81092e None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:49.131813 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 694/1386] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:23:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:49.138315 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4f35b-81fd-4511-b73b-29a55d8df2f4 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:49.138726 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4f35b-81fd-4511-b73b-29a55d8df2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:49.138880 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4f35b-81fd-4511-b73b-29a55d8df2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:49.207136 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-87b4f35b-81fd-4511-b73b-29a55d8df2f4 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:49.208349 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-87b4f35b-81fd-4511-b73b-29a55d8df2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:49.245845 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:49.317220 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 693/1387] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:23:49.373545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f5718-cbd7-46c4-b4c5-50e2bd81092e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:23:49.375627 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-408f5718-cbd7-46c4-b4c5-50e2bd81092e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:49.472385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:49.543506 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f5718-cbd7-46c4-b4c5-50e2bd81092e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:23:49.657707 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b88f0-f78a-43ca-a2d0-ceebc32dabef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:23:49.658461 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b88f0-f78a-43ca-a2d0-ceebc32dabef service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:49.658943 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b88f0-f78a-43ca-a2d0-ceebc32dabef service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:49.659222 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b88f0-f78a-43ca-a2d0-ceebc32dabef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:49.660871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-754b88f0-f78a-43ca-a2d0-ceebc32dabef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:49.700442 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:49.753168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b88f0-f78a-43ca-a2d0-ceebc32dabef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:23:49.754763 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b88f0-f78a-43ca-a2d0-ceebc32dabef service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:23:49.773952 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 695/1388] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:23:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:23:51.827258 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de3187-812a-43ff-8182-4760a7248251 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:51.827822 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de3187-812a-43ff-8182-4760a7248251 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:51.827822 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de3187-812a-43ff-8182-4760a7248251 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:51.829173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 694/1389] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:51.834935 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95130c-b45b-4e0e-bedf-65f528655544 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:51.835345 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95130c-b45b-4e0e-bedf-65f528655544 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:51.835489 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95130c-b45b-4e0e-bedf-65f528655544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:51.910849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2b95130c-b45b-4e0e-bedf-65f528655544 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:51.912064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2b95130c-b45b-4e0e-bedf-65f528655544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:52.008643 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 696/1390] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:23:52.052862 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de3187-812a-43ff-8182-4760a7248251 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:23:52.054405 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-54de3187-812a-43ff-8182-4760a7248251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:52.200859 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de3187-812a-43ff-8182-4760a7248251 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:23:52.216939 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:52.315874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419673a8-f424-4a18-8e03-2bdd86f163fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:23:52.316530 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419673a8-f424-4a18-8e03-2bdd86f163fe service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:52.316804 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419673a8-f424-4a18-8e03-2bdd86f163fe service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:52.317104 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419673a8-f424-4a18-8e03-2bdd86f163fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:52.318694 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-419673a8-f424-4a18-8e03-2bdd86f163fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:52.405678 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419673a8-f424-4a18-8e03-2bdd86f163fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:23:52.407267 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419673a8-f424-4a18-8e03-2bdd86f163fe service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:23:52.425278 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 695/1391] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:23:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:23:52.696891 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058a0d9-a8e6-4bed-83b0-7b13ddf49496 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:52.697178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058a0d9-a8e6-4bed-83b0-7b13ddf49496 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:52.697447 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058a0d9-a8e6-4bed-83b0-7b13ddf49496 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:52.699013 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 697/1392] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:52.703912 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474640ef-cc76-4d90-b827-6a0c9fd5ba38 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:52.704201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474640ef-cc76-4d90-b827-6a0c9fd5ba38 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:52.704517 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474640ef-cc76-4d90-b827-6a0c9fd5ba38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:52.731157 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:52.773087 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-474640ef-cc76-4d90-b827-6a0c9fd5ba38 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:52.774303 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-474640ef-cc76-4d90-b827-6a0c9fd5ba38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:52.870018 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 696/1393] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:23:52.925911 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934198b1-e429-44a7-a6d2-3aefc7b97885 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:52.926369 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934198b1-e429-44a7-a6d2-3aefc7b97885 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:52.926467 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934198b1-e429-44a7-a6d2-3aefc7b97885 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:52.927843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 698/1394] 10.222.0.20 () {54 vars in 966 bytes} [Sat Aug 6 12:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:52.933011 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93877cb5-b3ec-4825-81cf-1ca169d09025 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:52.933299 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93877cb5-b3ec-4825-81cf-1ca169d09025 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:52.933574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93877cb5-b3ec-4825-81cf-1ca169d09025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:53.010693 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-93877cb5-b3ec-4825-81cf-1ca169d09025 None None] MFA Rules not processed for user `7610d6289efb44599d0cee03b93f9938`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:53.011878 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-93877cb5-b3ec-4825-81cf-1ca169d09025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:53.051321 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:53.088154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 697/1395] 10.222.0.20 () {58 vars in 1063 bytes} [Sat Aug 6 12:23:52 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:23:53.095626 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934198b1-e429-44a7-a6d2-3aefc7b97885 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:23:53.097164 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-934198b1-e429-44a7-a6d2-3aefc7b97885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:53.111072 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:53.204383 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934198b1-e429-44a7-a6d2-3aefc7b97885 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:23:53.280736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40812640-93ac-4662-a612-6980395a906a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7610d6289efb44599d0cee03b93f9938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:23:53.281263 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40812640-93ac-4662-a612-6980395a906a service glance] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:53.281537 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40812640-93ac-4662-a612-6980395a906a service glance] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:53.281811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40812640-93ac-4662-a612-6980395a906a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:53.283422 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-40812640-93ac-4662-a612-6980395a906a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:53.351083 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:53.392675 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40812640-93ac-4662-a612-6980395a906a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:23:53.394126 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40812640-93ac-4662-a612-6980395a906a service glance] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:23:53.412160 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 699/1396] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:23:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:23:54.135831 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad272c-a883-4640-9d31-da1fecc6cb89 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:54.136354 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad272c-a883-4640-9d31-da1fecc6cb89 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:54.136468 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad272c-a883-4640-9d31-da1fecc6cb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:54.203478 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-22ad272c-a883-4640-9d31-da1fecc6cb89 None None] MFA Rules not processed for user `c810b52785b94c0cab4fd9f0d6e65825`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:54.204694 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-22ad272c-a883-4640-9d31-da1fecc6cb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:54.290629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 698/1397] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Aug 6 12:23:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:23:54.338492 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f386a2e-210e-40ec-a8f3-8e9a809c25b5 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:54.338950 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f386a2e-210e-40ec-a8f3-8e9a809c25b5 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:54.339054 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f386a2e-210e-40ec-a8f3-8e9a809c25b5 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:54.340428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 700/1398] 10.222.0.20 () {54 vars in 943 bytes} [Sat Aug 6 12:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:54.346527 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34f5d5-82fa-4b33-9638-1421d80b416e None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:54.346811 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34f5d5-82fa-4b33-9638-1421d80b416e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:54.347083 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34f5d5-82fa-4b33-9638-1421d80b416e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:54.414876 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:54.422992 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-be34f5d5-82fa-4b33-9638-1421d80b416e None None] MFA Rules not processed for user `2efc792a2655439f8b8070010059b6bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:54.424183 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-be34f5d5-82fa-4b33-9638-1421d80b416e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:54.460918 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0161bf5-793a-4dec-aa2b-bbe233127cb3 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:54.461203 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0161bf5-793a-4dec-aa2b-bbe233127cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:54.461482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0161bf5-793a-4dec-aa2b-bbe233127cb3 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:54.462847 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 701/1399] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:23:54.467749 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d4907-f1bb-4c4d-a3ff-6e4ef4dd1140 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:54.468583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d4907-f1bb-4c4d-a3ff-6e4ef4dd1140 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:54.468583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d4907-f1bb-4c4d-a3ff-6e4ef4dd1140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:54.534854 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-fc9d4907-f1bb-4c4d-a3ff-6e4ef4dd1140 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:54.535367 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 699/1400] 10.222.0.20 () {58 vars in 1040 bytes} [Sat Aug 6 12:23:54 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:23:54.536067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fc9d4907-f1bb-4c4d-a3ff-6e4ef4dd1140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:54.544724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34f5d5-82fa-4b33-9638-1421d80b416e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:23:54.554623 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34f5d5-82fa-4b33-9638-1421d80b416e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:23:54.635216 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 702/1401] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:23:54.664813 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2dc156-f618-4f10-b702-3f5ebb13fd8f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efc792a2655439f8b8070010059b6bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:23:54.665572 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2dc156-f618-4f10-b702-3f5ebb13fd8f service swift] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:54.666002 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2dc156-f618-4f10-b702-3f5ebb13fd8f service swift] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:54.666284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2dc156-f618-4f10-b702-3f5ebb13fd8f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:54.667956 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2f2dc156-f618-4f10-b702-3f5ebb13fd8f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:54.670520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:54.679890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d4907-f1bb-4c4d-a3ff-6e4ef4dd1140 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:23:54.681226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fc9d4907-f1bb-4c4d-a3ff-6e4ef4dd1140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:54.711572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:54.760090 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2dc156-f618-4f10-b702-3f5ebb13fd8f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:23:54.761721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2dc156-f618-4f10-b702-3f5ebb13fd8f service swift] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:23:54.769767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 700/1402] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Aug 6 12:23:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 06 12:23:54.809259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d4907-f1bb-4c4d-a3ff-6e4ef4dd1140 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:23:54.934839 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69459b46-196c-4ec7-9432-4866cd93c14e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:23:54.935352 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69459b46-196c-4ec7-9432-4866cd93c14e service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:54.935628 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69459b46-196c-4ec7-9432-4866cd93c14e service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:54.935894 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69459b46-196c-4ec7-9432-4866cd93c14e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:54.937558 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-69459b46-196c-4ec7-9432-4866cd93c14e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:54.955721 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:55.031662 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69459b46-196c-4ec7-9432-4866cd93c14e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:23:55.033300 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69459b46-196c-4ec7-9432-4866cd93c14e service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:23:55.051455 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 703/1403] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:23:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:23:57.132851 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b02b20-38bb-4b75-b5dd-803b6de69c86 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:57.133415 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b02b20-38bb-4b75-b5dd-803b6de69c86 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:57.133415 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b02b20-38bb-4b75-b5dd-803b6de69c86 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:57.134755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 701/1404] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:57.140052 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1ed26-e9b8-4f17-902d-9715acac40e1 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:57.140528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1ed26-e9b8-4f17-902d-9715acac40e1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:57.140817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1ed26-e9b8-4f17-902d-9715acac40e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:57.214350 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-69f1ed26-e9b8-4f17-902d-9715acac40e1 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:57.215713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-69f1ed26-e9b8-4f17-902d-9715acac40e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:57.319777 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:57.327210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 704/1405] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:23:57.373068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b02b20-38bb-4b75-b5dd-803b6de69c86 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:23:57.374598 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a0b02b20-38bb-4b75-b5dd-803b6de69c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:57.531181 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:57.533521 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b02b20-38bb-4b75-b5dd-803b6de69c86 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:23:57.655541 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344512c-006a-4a39-8af6-cf548966271e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:23:57.656776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344512c-006a-4a39-8af6-cf548966271e service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:57.657170 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344512c-006a-4a39-8af6-cf548966271e service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:57.657554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344512c-006a-4a39-8af6-cf548966271e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:57.659672 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8344512c-006a-4a39-8af6-cf548966271e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:23:57.763922 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344512c-006a-4a39-8af6-cf548966271e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:23:57.765563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344512c-006a-4a39-8af6-cf548966271e service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:23:57.785201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 702/1406] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:23:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:23:59.828828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d3ac1-4c40-4ea4-865d-6c37fcf23678 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:59.829388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d3ac1-4c40-4ea4-865d-6c37fcf23678 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:59.829388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d3ac1-4c40-4ea4-865d-6c37fcf23678 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:59.830732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 705/1407] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:23:59.835882 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5e066-184d-40a0-9ae1-033ded434626 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:23:59.836282 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5e066-184d-40a0-9ae1-033ded434626 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:23:59.836488 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5e066-184d-40a0-9ae1-033ded434626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:23:59.840976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:23:59.909700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ddf5e066-184d-40a0-9ae1-033ded434626 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:23:59.911541 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ddf5e066-184d-40a0-9ae1-033ded434626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:00.030877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 703/1408] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:23:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:24:00.078597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d3ac1-4c40-4ea4-865d-6c37fcf23678 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:24:00.080138 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-706d3ac1-4c40-4ea4-865d-6c37fcf23678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:00.243094 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d3ac1-4c40-4ea4-865d-6c37fcf23678 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:24:00.320951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:00.372110 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79a1ca-e526-4971-953a-52ba31905e95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:24:00.372675 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79a1ca-e526-4971-953a-52ba31905e95 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:00.372948 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79a1ca-e526-4971-953a-52ba31905e95 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:00.373218 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79a1ca-e526-4971-953a-52ba31905e95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:00.374830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7d79a1ca-e526-4971-953a-52ba31905e95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:00.476426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79a1ca-e526-4971-953a-52ba31905e95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:24:00.478094 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79a1ca-e526-4971-953a-52ba31905e95 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:24:00.497622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 706/1409] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:24:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:24:10.028754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=64 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:10.494760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=64 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:17.134713 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993fbbc-f859-4841-8ab1-29c92ab017dc None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:17.135240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993fbbc-f859-4841-8ab1-29c92ab017dc None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:17.135240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993fbbc-f859-4841-8ab1-29c92ab017dc None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:17.136550 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 704/1410] 10.222.0.20 () {54 vars in 918 bytes} [Sat Aug 6 12:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:24:17.141559 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae1036-d515-4f3d-a992-2cd0701a0e1e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:17.142011 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae1036-d515-4f3d-a992-2cd0701a0e1e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:17.142107 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae1036-d515-4f3d-a992-2cd0701a0e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:17.218300 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-95ae1036-d515-4f3d-a992-2cd0701a0e1e None None] MFA Rules not processed for user `e2244655ba304b9c980b5e68772eefc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:24:17.219506 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-95ae1036-d515-4f3d-a992-2cd0701a0e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:17.323770 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 707/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Sat Aug 6 12:24:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:24:17.338434 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6658fe-214c-43aa-99e2-c8fca127381e None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:17.338720 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6658fe-214c-43aa-99e2-c8fca127381e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:17.338996 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6658fe-214c-43aa-99e2-c8fca127381e None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:17.340370 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 705/1412] 10.222.0.20 () {54 vars in 918 bytes} [Sat Aug 6 12:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:24:17.344727 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5d858-08ce-4f88-982e-25263f64f4af None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:17.345014 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5d858-08ce-4f88-982e-25263f64f4af None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:17.345288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5d858-08ce-4f88-982e-25263f64f4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:17.416868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:17.419441 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5df5d858-08ce-4f88-982e-25263f64f4af None None] MFA Rules not processed for user `524b728bf63c4ca88e0f25e531cf5056`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:24:17.421203 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5df5d858-08ce-4f88-982e-25263f64f4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:17.536672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 708/1413] 10.222.0.20 () {58 vars in 1015 bytes} [Sat Aug 6 12:24:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:24:19.982693 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc1c6b-79db-41a4-988e-1ca54920d496 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:19.983216 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc1c6b-79db-41a4-988e-1ca54920d496 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:19.983216 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc1c6b-79db-41a4-988e-1ca54920d496 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:19.984864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 12:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:24:19.990016 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c34e0-1ce6-4861-ba28-47790622a21b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:19.990503 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c34e0-1ce6-4861-ba28-47790622a21b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:19.990696 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c34e0-1ce6-4861-ba28-47790622a21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:20.055117 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d92c34e0-1ce6-4861-ba28-47790622a21b None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:24:20.056386 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d92c34e0-1ce6-4861-ba28-47790622a21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:20.111466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:20.152824 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 12:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:24:20.195179 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc1c6b-79db-41a4-988e-1ca54920d496 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:24:20.196777 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-73cc1c6b-79db-41a4-988e-1ca54920d496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:20.339786 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc1c6b-79db-41a4-988e-1ca54920d496 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:24:20.409690 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:20.458056 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea21350-ccde-4b9f-83c0-4dfc4fe8db76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:24:20.458668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea21350-ccde-4b9f-83c0-4dfc4fe8db76 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:20.458948 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea21350-ccde-4b9f-83c0-4dfc4fe8db76 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:20.459219 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea21350-ccde-4b9f-83c0-4dfc4fe8db76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:20.460885 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fea21350-ccde-4b9f-83c0-4dfc4fe8db76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:20.550057 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea21350-ccde-4b9f-83c0-4dfc4fe8db76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:24:20.551656 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea21350-ccde-4b9f-83c0-4dfc4fe8db76 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:24:20.569522 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 707/1416] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:24:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:24:21.459387 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d1189-f923-4951-aae8-caf73794a593 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:21.459951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d1189-f923-4951-aae8-caf73794a593 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:21.459951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d1189-f923-4951-aae8-caf73794a593 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:21.461331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 12:24:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:24:21.466292 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c9358-d34b-432d-81bd-c18b99d91372 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:21.466695 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c9358-d34b-432d-81bd-c18b99d91372 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:21.466835 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c9358-d34b-432d-81bd-c18b99d91372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:21.541029 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-187c9358-d34b-432d-81bd-c18b99d91372 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:24:21.542226 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-187c9358-d34b-432d-81bd-c18b99d91372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:21.568584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:21.637960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 12:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:24:21.787163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d1189-f923-4951-aae8-caf73794a593 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:24:21.788785 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4e7d1189-f923-4951-aae8-caf73794a593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:21.930285 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d1189-f923-4951-aae8-caf73794a593 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:24:21.969585 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:22.043429 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7eec6-784a-4e5f-ba5d-d6b709fbfde2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:24:22.043980 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7eec6-784a-4e5f-ba5d-d6b709fbfde2 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:22.044261 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7eec6-784a-4e5f-ba5d-d6b709fbfde2 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:22.044567 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7eec6-784a-4e5f-ba5d-d6b709fbfde2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:22.046190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a6e7eec6-784a-4e5f-ba5d-d6b709fbfde2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:22.136054 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7eec6-784a-4e5f-ba5d-d6b709fbfde2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:24:22.137685 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7eec6-784a-4e5f-ba5d-d6b709fbfde2 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:24:22.155312 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 711/1419] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:24:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:24:23.068881 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da84b1-e706-4476-b90d-9a477c5a6023 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:23.069444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da84b1-e706-4476-b90d-9a477c5a6023 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:23.069444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da84b1-e706-4476-b90d-9a477c5a6023 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:23.070754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 12:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:24:23.075657 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1885fe-7f8d-4afa-9a51-afba6527bcef None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:23.076112 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1885fe-7f8d-4afa-9a51-afba6527bcef None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:23.076221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1885fe-7f8d-4afa-9a51-afba6527bcef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:23.137840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:23.167237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-bd1885fe-7f8d-4afa-9a51-afba6527bcef None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:24:23.168498 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bd1885fe-7f8d-4afa-9a51-afba6527bcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:23.263004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 12:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:24:23.414551 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da84b1-e706-4476-b90d-9a477c5a6023 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:24:23.416058 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c0da84b1-e706-4476-b90d-9a477c5a6023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:23.558151 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da84b1-e706-4476-b90d-9a477c5a6023 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:24:23.569921 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:23.677852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91f806-4dab-42c3-aad8-eba573aa2bb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:24:23.678467 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91f806-4dab-42c3-aad8-eba573aa2bb4 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:23.678749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91f806-4dab-42c3-aad8-eba573aa2bb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:23.679023 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91f806-4dab-42c3-aad8-eba573aa2bb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:23.680680 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6c91f806-4dab-42c3-aad8-eba573aa2bb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:23.767184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91f806-4dab-42c3-aad8-eba573aa2bb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:24:23.768814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91f806-4dab-42c3-aad8-eba573aa2bb4 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:24:23.786949 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 710/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:24:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:24:24.641379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e19e8e-d074-4fc3-b169-850b70105640 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:24.641922 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e19e8e-d074-4fc3-b169-850b70105640 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:24.641922 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e19e8e-d074-4fc3-b169-850b70105640 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:24.643278 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 12:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:24:24.648841 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4290e-d5ab-4538-b798-e406ff393b11 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:24.649263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4290e-d5ab-4538-b798-e406ff393b11 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:24.649373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4290e-d5ab-4538-b798-e406ff393b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:24.669636 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:24.726519 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-89d4290e-d5ab-4538-b798-e406ff393b11 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:24:24.727709 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-89d4290e-d5ab-4538-b798-e406ff393b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:24.824980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 12:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:24:24.974242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e19e8e-d074-4fc3-b169-850b70105640 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:24:24.975744 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b6e19e8e-d074-4fc3-b169-850b70105640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:25.084957 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:25.121116 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e19e8e-d074-4fc3-b169-850b70105640 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:24:25.232278 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1eda7-9372-4812-99ec-99b5032bab10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:24:25.232921 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1eda7-9372-4812-99ec-99b5032bab10 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:25.233199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1eda7-9372-4812-99ec-99b5032bab10 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:25.233469 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1eda7-9372-4812-99ec-99b5032bab10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:25.235333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bfa1eda7-9372-4812-99ec-99b5032bab10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:25.308284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:25.322228 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1eda7-9372-4812-99ec-99b5032bab10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:24:25.323805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1eda7-9372-4812-99ec-99b5032bab10 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:24:25.341808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 714/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:24:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:24:26.199464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e6b14-ff5a-4d55-91a7-7ae136732fef None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:26.199988 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e6b14-ff5a-4d55-91a7-7ae136732fef None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:26.199988 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e6b14-ff5a-4d55-91a7-7ae136732fef None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:26.201392 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 12:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:24:26.206412 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52f0de-e1ae-45f3-9f81-280f813520f3 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:26.206846 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52f0de-e1ae-45f3-9f81-280f813520f3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:26.206940 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52f0de-e1ae-45f3-9f81-280f813520f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:26.283945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-df52f0de-e1ae-45f3-9f81-280f813520f3 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:24:26.285239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-df52f0de-e1ae-45f3-9f81-280f813520f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:26.384906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 12:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:24:26.533834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e6b14-ff5a-4d55-91a7-7ae136732fef None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:24:26.535414 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b77e6b14-ff5a-4d55-91a7-7ae136732fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:26.618181 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:26.680672 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e6b14-ff5a-4d55-91a7-7ae136732fef None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:24:26.790711 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2c147-1cf1-416d-a52c-2144f64db7a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:24:26.791247 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2c147-1cf1-416d-a52c-2144f64db7a0 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:26.791525 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2c147-1cf1-416d-a52c-2144f64db7a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:26.791799 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2c147-1cf1-416d-a52c-2144f64db7a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:26.793445 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-30b2c147-1cf1-416d-a52c-2144f64db7a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:26.840062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:26.881714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2c147-1cf1-416d-a52c-2144f64db7a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:24:26.883301 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2c147-1cf1-416d-a52c-2144f64db7a0 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:24:26.901005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 713/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:24:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:24:27.698350 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2fba7-5299-43ea-8d67-73ba6daf4a12 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:27.698926 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2fba7-5299-43ea-8d67-73ba6daf4a12 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:27.698926 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2fba7-5299-43ea-8d67-73ba6daf4a12 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:27.700264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 12:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:24:27.705202 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef36bdb-53b0-4159-a888-369f9ba702e3 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:27.705644 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef36bdb-53b0-4159-a888-369f9ba702e3 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:27.705752 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef36bdb-53b0-4159-a888-369f9ba702e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:27.779738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3ef36bdb-53b0-4159-a888-369f9ba702e3 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:24:27.780968 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3ef36bdb-53b0-4159-a888-369f9ba702e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:27.878654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 12:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:24:28.029187 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2fba7-5299-43ea-8d67-73ba6daf4a12 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:24:28.030703 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-58e2fba7-5299-43ea-8d67-73ba6daf4a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:28.079353 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:28.177495 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2fba7-5299-43ea-8d67-73ba6daf4a12 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:24:28.286428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e47d9-d553-43fe-aeef-ba1d880cff7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:24:28.287031 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e47d9-d553-43fe-aeef-ba1d880cff7d service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:28.287308 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e47d9-d553-43fe-aeef-ba1d880cff7d service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:28.287579 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e47d9-d553-43fe-aeef-ba1d880cff7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:28.289220 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-122e47d9-d553-43fe-aeef-ba1d880cff7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:28.296161 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:28.377829 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e47d9-d553-43fe-aeef-ba1d880cff7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:24:28.379417 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e47d9-d553-43fe-aeef-ba1d880cff7d service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:24:28.397108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 717/1431] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:24:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:24:29.195974 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02014e2-82bc-43f7-a81c-5ec276504fe1 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:29.196583 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02014e2-82bc-43f7-a81c-5ec276504fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:29.196583 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02014e2-82bc-43f7-a81c-5ec276504fe1 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:29.197917 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 12:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:24:29.202817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe163807-9b3c-4fda-a8b2-54b80ab531e2 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:29.203243 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe163807-9b3c-4fda-a8b2-54b80ab531e2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:29.203355 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe163807-9b3c-4fda-a8b2-54b80ab531e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:29.279053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-fe163807-9b3c-4fda-a8b2-54b80ab531e2 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:24:29.280177 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fe163807-9b3c-4fda-a8b2-54b80ab531e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:29.345707 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:29.375007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 12:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:24:29.524789 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02014e2-82bc-43f7-a81c-5ec276504fe1 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:24:29.526320 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f02014e2-82bc-43f7-a81c-5ec276504fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:29.544033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:29.668943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02014e2-82bc-43f7-a81c-5ec276504fe1 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:24:29.753287 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:29.779410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7418ebf0-68c3-4586-a989-de5fad3869c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:24:29.780043 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418ebf0-68c3-4586-a989-de5fad3869c1 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:29.780340 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418ebf0-68c3-4586-a989-de5fad3869c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:29.780615 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418ebf0-68c3-4586-a989-de5fad3869c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:29.782213 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7418ebf0-68c3-4586-a989-de5fad3869c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:29.870246 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7418ebf0-68c3-4586-a989-de5fad3869c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:24:29.871776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7418ebf0-68c3-4586-a989-de5fad3869c1 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:24:29.889937 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 716/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:24:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:24:30.703352 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492987d-f5cb-4aaf-9668-c6ba5d3dd4d1 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:30.703881 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492987d-f5cb-4aaf-9668-c6ba5d3dd4d1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:30.703881 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492987d-f5cb-4aaf-9668-c6ba5d3dd4d1 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:30.705234 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 12:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:24:30.710653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896e973-0d6f-41bc-8c2d-ef6845077170 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:30.711080 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896e973-0d6f-41bc-8c2d-ef6845077170 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:30.711195 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896e973-0d6f-41bc-8c2d-ef6845077170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:30.785637 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-4896e973-0d6f-41bc-8c2d-ef6845077170 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:24:30.786827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4896e973-0d6f-41bc-8c2d-ef6845077170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:30.827719 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:30.892531 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 12:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:24:31.041940 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492987d-f5cb-4aaf-9668-c6ba5d3dd4d1 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:24:31.043581 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a492987d-f5cb-4aaf-9668-c6ba5d3dd4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:31.187676 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492987d-f5cb-4aaf-9668-c6ba5d3dd4d1 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:24:31.241653 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:31.300882 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8718e2-4365-4b24-b08b-e9a2ac09e432 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:24:31.301531 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8718e2-4365-4b24-b08b-e9a2ac09e432 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:31.301816 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8718e2-4365-4b24-b08b-e9a2ac09e432 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:31.302087 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8718e2-4365-4b24-b08b-e9a2ac09e432 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:31.303697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ff8718e2-4365-4b24-b08b-e9a2ac09e432 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:31.392861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8718e2-4365-4b24-b08b-e9a2ac09e432 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:24:31.394444 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8718e2-4365-4b24-b08b-e9a2ac09e432 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:24:31.412683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 720/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:24:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:24:32.209765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454346d2-ed43-4ac9-bf00-415e2588112e None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:32.210295 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454346d2-ed43-4ac9-bf00-415e2588112e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:32.210408 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454346d2-ed43-4ac9-bf00-415e2588112e None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:32.211696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 12:24:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:24:32.216522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8779247-30cc-48d2-b130-d841cbe19ab1 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:32.216914 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8779247-30cc-48d2-b130-d841cbe19ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:32.217068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8779247-30cc-48d2-b130-d841cbe19ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:32.409406 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e8779247-30cc-48d2-b130-d841cbe19ab1 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:24:32.410591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e8779247-30cc-48d2-b130-d841cbe19ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:32.412598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:32.505099 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 12:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:24:32.658641 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454346d2-ed43-4ac9-bf00-415e2588112e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:24:32.660358 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-454346d2-ed43-4ac9-bf00-415e2588112e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:32.921280 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454346d2-ed43-4ac9-bf00-415e2588112e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:24:32.947361 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:33.034169 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b434f-adf9-49e5-9deb-824663bacb56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:24:33.034786 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b434f-adf9-49e5-9deb-824663bacb56 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:33.035065 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b434f-adf9-49e5-9deb-824663bacb56 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:33.035334 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b434f-adf9-49e5-9deb-824663bacb56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:33.037611 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fb7b434f-adf9-49e5-9deb-824663bacb56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:33.123548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b434f-adf9-49e5-9deb-824663bacb56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:24:33.125163 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b434f-adf9-49e5-9deb-824663bacb56 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:24:33.143268 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 719/1440] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:24:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:24:35.056570 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0fa90-d9ec-4b2a-965f-9a669fc5d905 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:35.057121 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0fa90-d9ec-4b2a-965f-9a669fc5d905 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:35.057121 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0fa90-d9ec-4b2a-965f-9a669fc5d905 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:35.058489 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 12:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:24:35.063784 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c468c7-54f9-40fc-ba9d-a851893e962e None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:35.064205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c468c7-54f9-40fc-ba9d-a851893e962e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:35.064355 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c468c7-54f9-40fc-ba9d-a851893e962e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:35.105550 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:35.132166 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-d4c468c7-54f9-40fc-ba9d-a851893e962e None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:24:35.133429 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d4c468c7-54f9-40fc-ba9d-a851893e962e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:35.229986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 12:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:24:35.270955 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0fa90-d9ec-4b2a-965f-9a669fc5d905 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:24:35.272587 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e1a0fa90-d9ec-4b2a-965f-9a669fc5d905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:35.373008 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0fa90-d9ec-4b2a-965f-9a669fc5d905 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:24:35.403891 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:35.444772 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba8173-00b0-4f72-931c-86c5e8dec082 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:24:35.445381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba8173-00b0-4f72-931c-86c5e8dec082 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:35.445661 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba8173-00b0-4f72-931c-86c5e8dec082 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:35.445934 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba8173-00b0-4f72-931c-86c5e8dec082 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:35.447538 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1aba8173-00b0-4f72-931c-86c5e8dec082 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:35.555417 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba8173-00b0-4f72-931c-86c5e8dec082 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:24:35.556869 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba8173-00b0-4f72-931c-86c5e8dec082 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:24:35.574668 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 723/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:24:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:24:37.715234 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd6d10-c5b9-4e74-a624-1e97f905f400 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:37.715816 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd6d10-c5b9-4e74-a624-1e97f905f400 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:37.715816 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd6d10-c5b9-4e74-a624-1e97f905f400 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:37.717154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 12:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:24:37.722348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661baa7b-9e10-4f56-bf29-72249345ba05 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:37.722756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661baa7b-9e10-4f56-bf29-72249345ba05 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:37.722923 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661baa7b-9e10-4f56-bf29-72249345ba05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:37.787448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-661baa7b-9e10-4f56-bf29-72249345ba05 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:24:37.788679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-661baa7b-9e10-4f56-bf29-72249345ba05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:37.801069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:37.883873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 12:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:24:37.921907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd6d10-c5b9-4e74-a624-1e97f905f400 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:24:37.923417 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d1dd6d10-c5b9-4e74-a624-1e97f905f400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:38.025174 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd6d10-c5b9-4e74-a624-1e97f905f400 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:24:38.037372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:38.096710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9920519-1f68-4f0f-9f22-66817beddbab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:24:38.097316 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9920519-1f68-4f0f-9f22-66817beddbab service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:38.097593 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9920519-1f68-4f0f-9f22-66817beddbab service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:38.097857 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9920519-1f68-4f0f-9f22-66817beddbab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:38.100118 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c9920519-1f68-4f0f-9f22-66817beddbab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:38.207049 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9920519-1f68-4f0f-9f22-66817beddbab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:24:38.208504 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9920519-1f68-4f0f-9f22-66817beddbab service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:24:38.231804 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 722/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:24:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:24:40.562154 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982550c8-dc0b-4511-bdfa-5d13deda3fe5 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:40.562625 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982550c8-dc0b-4511-bdfa-5d13deda3fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:40.562717 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982550c8-dc0b-4511-bdfa-5d13deda3fe5 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:40.564068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 12:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:24:40.569339 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c50fba-cb9c-4bd8-9079-23a0f3e4017f None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:40.569741 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c50fba-cb9c-4bd8-9079-23a0f3e4017f None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:40.569893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c50fba-cb9c-4bd8-9079-23a0f3e4017f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:40.617703 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:40.635302 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-20c50fba-cb9c-4bd8-9079-23a0f3e4017f None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:24:40.636516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-20c50fba-cb9c-4bd8-9079-23a0f3e4017f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:40.731487 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 12:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:24:40.770007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982550c8-dc0b-4511-bdfa-5d13deda3fe5 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:24:40.772047 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-982550c8-dc0b-4511-bdfa-5d13deda3fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:40.873471 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982550c8-dc0b-4511-bdfa-5d13deda3fe5 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:24:40.913607 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:40.947886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876088d-5e09-468b-acf4-254ea6c36811 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:24:40.948526 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876088d-5e09-468b-acf4-254ea6c36811 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:40.948803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876088d-5e09-468b-acf4-254ea6c36811 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:40.949074 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876088d-5e09-468b-acf4-254ea6c36811 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:40.950710 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8876088d-5e09-468b-acf4-254ea6c36811 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:41.058000 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876088d-5e09-468b-acf4-254ea6c36811 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:24:41.059431 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876088d-5e09-468b-acf4-254ea6c36811 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:24:41.077529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 726/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:24:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:24:43.169518 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30fd1bd-f15b-40fc-a556-08ec8e3017d4 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:43.170074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30fd1bd-f15b-40fc-a556-08ec8e3017d4 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:43.170074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30fd1bd-f15b-40fc-a556-08ec8e3017d4 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:43.171444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 12:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:24:43.176543 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06c203-3ae9-4275-8d89-368808931ce7 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:43.176976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06c203-3ae9-4275-8d89-368808931ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:43.177081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06c203-3ae9-4275-8d89-368808931ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:43.242022 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-dd06c203-3ae9-4275-8d89-368808931ce7 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:24:43.243205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-dd06c203-3ae9-4275-8d89-368808931ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:43.263342 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:43.339019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 12:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:24:43.377446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30fd1bd-f15b-40fc-a556-08ec8e3017d4 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:24:43.378959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f30fd1bd-f15b-40fc-a556-08ec8e3017d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:43.487528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30fd1bd-f15b-40fc-a556-08ec8e3017d4 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:24:43.507889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:43.559680 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5253bd8-f107-4c64-9e9b-70074335f751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:24:43.560287 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5253bd8-f107-4c64-9e9b-70074335f751 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:43.560595 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5253bd8-f107-4c64-9e9b-70074335f751 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:43.560870 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5253bd8-f107-4c64-9e9b-70074335f751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:43.562467 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a5253bd8-f107-4c64-9e9b-70074335f751 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:43.669911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5253bd8-f107-4c64-9e9b-70074335f751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:24:43.671332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5253bd8-f107-4c64-9e9b-70074335f751 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:24:43.689338 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 725/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:24:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:24:45.746547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932b679-da05-468c-afd1-43521ad89a62 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:45.747118 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932b679-da05-468c-afd1-43521ad89a62 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:45.747118 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932b679-da05-468c-afd1-43521ad89a62 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:45.748457 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 12:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:24:45.753597 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fd699-08ea-43c1-ba94-19958e6ba732 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:45.754002 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fd699-08ea-43c1-ba94-19958e6ba732 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:45.754146 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fd699-08ea-43c1-ba94-19958e6ba732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:45.817970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:45.820920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-633fd699-08ea-43c1-ba94-19958e6ba732 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:24:45.822759 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-633fd699-08ea-43c1-ba94-19958e6ba732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:45.918573 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 12:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:24:45.957719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932b679-da05-468c-afd1-43521ad89a62 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:24:45.959220 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4932b679-da05-468c-afd1-43521ad89a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:46.060818 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932b679-da05-468c-afd1-43521ad89a62 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:24:46.108434 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:46.133098 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a9375-0948-47b8-908f-274e81d164c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:24:46.133704 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a9375-0948-47b8-908f-274e81d164c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:46.133979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a9375-0948-47b8-908f-274e81d164c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:46.134248 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a9375-0948-47b8-908f-274e81d164c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:46.135838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-887a9375-0948-47b8-908f-274e81d164c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:46.243831 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a9375-0948-47b8-908f-274e81d164c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:24:46.245283 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a9375-0948-47b8-908f-274e81d164c1 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:24:46.263908 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 729/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:24:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:24:48.321830 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf230414-8480-489a-b239-7ed0f554c5c6 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:48.322383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf230414-8480-489a-b239-7ed0f554c5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:48.322383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf230414-8480-489a-b239-7ed0f554c5c6 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:48.323727 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 12:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:24:48.328984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56d867-3aef-433c-9da5-fdda83014d5a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:48.329434 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56d867-3aef-433c-9da5-fdda83014d5a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:48.329572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56d867-3aef-433c-9da5-fdda83014d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:48.396520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5e56d867-3aef-433c-9da5-fdda83014d5a None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:24:48.397717 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5e56d867-3aef-433c-9da5-fdda83014d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:48.427125 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:48.496735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 12:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:24:48.535138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf230414-8480-489a-b239-7ed0f554c5c6 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:24:48.536710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-cf230414-8480-489a-b239-7ed0f554c5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:48.640800 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf230414-8480-489a-b239-7ed0f554c5c6 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:24:48.670105 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:48.714527 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d8085-8584-431c-95e2-064cd89be3c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:24:48.715126 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d8085-8584-431c-95e2-064cd89be3c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:48.715401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d8085-8584-431c-95e2-064cd89be3c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:48.715671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d8085-8584-431c-95e2-064cd89be3c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:48.717300 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4b2d8085-8584-431c-95e2-064cd89be3c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:48.823282 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2d8085-8584-431c-95e2-064cd89be3c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:24:48.824727 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2d8085-8584-431c-95e2-064cd89be3c2 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:24:48.841844 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 728/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:24:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:24:50.897019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b3c8f-08e2-433e-98ec-b40ea3dd7eaf None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:50.897589 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b3c8f-08e2-433e-98ec-b40ea3dd7eaf None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:50.897589 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b3c8f-08e2-433e-98ec-b40ea3dd7eaf None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:50.898927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 12:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:24:50.904156 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc1bd5-229f-4a00-b1a9-bcfda1f5c599 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:50.904602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc1bd5-229f-4a00-b1a9-bcfda1f5c599 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:50.904765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc1bd5-229f-4a00-b1a9-bcfda1f5c599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:50.969416 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-08dc1bd5-229f-4a00-b1a9-bcfda1f5c599 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:24:50.970605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-08dc1bd5-229f-4a00-b1a9-bcfda1f5c599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:50.978746 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:51.066330 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 12:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:24:51.104881 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b3c8f-08e2-433e-98ec-b40ea3dd7eaf None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:24:51.106391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-405b3c8f-08e2-433e-98ec-b40ea3dd7eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:51.211130 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b3c8f-08e2-433e-98ec-b40ea3dd7eaf None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:24:51.265101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:51.283445 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55fe232-3613-4551-9a65-36d9e053f88e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:24:51.284052 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fe232-3613-4551-9a65-36d9e053f88e service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:51.284356 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fe232-3613-4551-9a65-36d9e053f88e service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:51.284633 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fe232-3613-4551-9a65-36d9e053f88e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:51.286843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f55fe232-3613-4551-9a65-36d9e053f88e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:51.394628 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55fe232-3613-4551-9a65-36d9e053f88e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:24:51.396666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55fe232-3613-4551-9a65-36d9e053f88e service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:24:51.414347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 732/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:24:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:24:53.449094 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882ece2-c37d-4bf9-be08-6c0ebdca0fdd None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:53.449654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882ece2-c37d-4bf9-be08-6c0ebdca0fdd None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:53.449654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882ece2-c37d-4bf9-be08-6c0ebdca0fdd None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:53.451021 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 12:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:24:53.456362 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2e4ca-c9ed-43f9-b7bd-87c6113cfe81 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:53.456809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2e4ca-c9ed-43f9-b7bd-87c6113cfe81 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:53.456908 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2e4ca-c9ed-43f9-b7bd-87c6113cfe81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:53.524267 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-90f2e4ca-c9ed-43f9-b7bd-87c6113cfe81 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:24:53.525510 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-90f2e4ca-c9ed-43f9-b7bd-87c6113cfe81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:53.567382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:53.626954 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 12:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:24:53.664847 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882ece2-c37d-4bf9-be08-6c0ebdca0fdd None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:24:53.666356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5882ece2-c37d-4bf9-be08-6c0ebdca0fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:53.766723 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882ece2-c37d-4bf9-be08-6c0ebdca0fdd None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:24:53.803782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:53.839384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4801e0-9611-4765-b76d-525eadd44f91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:24:53.839992 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4801e0-9611-4765-b76d-525eadd44f91 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:53.840266 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4801e0-9611-4765-b76d-525eadd44f91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:53.840563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4801e0-9611-4765-b76d-525eadd44f91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:53.842166 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ca4801e0-9611-4765-b76d-525eadd44f91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:53.948574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4801e0-9611-4765-b76d-525eadd44f91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:24:53.949975 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4801e0-9611-4765-b76d-525eadd44f91 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:24:53.967883 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 731/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:24:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:24:55.956549 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cba9f-7051-4620-8c94-b8f599fb3cd8 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:55.957147 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cba9f-7051-4620-8c94-b8f599fb3cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:55.957147 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cba9f-7051-4620-8c94-b8f599fb3cd8 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:55.959031 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 12:24:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:24:55.964351 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10c005-1392-4e92-aa24-49534cf82fe4 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:55.964773 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10c005-1392-4e92-aa24-49534cf82fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:55.964904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10c005-1392-4e92-aa24-49534cf82fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:56.031915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-bc10c005-1392-4e92-aa24-49534cf82fe4 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:24:56.033136 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-bc10c005-1392-4e92-aa24-49534cf82fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:56.050868 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:56.128458 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 12:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:24:56.165347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17cba9f-7051-4620-8c94-b8f599fb3cd8 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:24:56.166919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a17cba9f-7051-4620-8c94-b8f599fb3cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:56.269865 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17cba9f-7051-4620-8c94-b8f599fb3cd8 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:24:56.334518 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:56.341651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6c6b1-9525-48c3-9f8c-d4fd1a21585c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:24:56.342259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6c6b1-9525-48c3-9f8c-d4fd1a21585c service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:56.342541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6c6b1-9525-48c3-9f8c-d4fd1a21585c service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:56.342814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6c6b1-9525-48c3-9f8c-d4fd1a21585c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:56.344451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e0a6c6b1-9525-48c3-9f8c-d4fd1a21585c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:56.457375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a6c6b1-9525-48c3-9f8c-d4fd1a21585c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:24:56.458847 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a6c6b1-9525-48c3-9f8c-d4fd1a21585c service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:24:56.476633 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 735/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:24:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:24:58.448833 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60fb6a-b603-49aa-affc-0a67f7e292b6 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:58.449395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60fb6a-b603-49aa-affc-0a67f7e292b6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:58.449395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60fb6a-b603-49aa-affc-0a67f7e292b6 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:58.450728 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 12:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:24:58.455809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53714fa-73ef-4ac3-ac02-9ae019760c39 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:58.456207 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53714fa-73ef-4ac3-ac02-9ae019760c39 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:58.456407 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53714fa-73ef-4ac3-ac02-9ae019760c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:58.524005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e53714fa-73ef-4ac3-ac02-9ae019760c39 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:24:58.525243 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e53714fa-73ef-4ac3-ac02-9ae019760c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:58.573178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:58.620561 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 12:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:24:58.657058 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60fb6a-b603-49aa-affc-0a67f7e292b6 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:24:58.658560 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5d60fb6a-b603-49aa-affc-0a67f7e292b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:58.761400 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60fb6a-b603-49aa-affc-0a67f7e292b6 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:24:58.807841 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:24:58.834367 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9575b18-7c75-46ab-8aca-3b8f3a34df44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:24:58.834964 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9575b18-7c75-46ab-8aca-3b8f3a34df44 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:24:58.835239 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9575b18-7c75-46ab-8aca-3b8f3a34df44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:24:58.835507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9575b18-7c75-46ab-8aca-3b8f3a34df44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:24:58.837133 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e9575b18-7c75-46ab-8aca-3b8f3a34df44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:24:58.941657 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9575b18-7c75-46ab-8aca-3b8f3a34df44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:24:58.943099 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9575b18-7c75-46ab-8aca-3b8f3a34df44 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:24:58.960720 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 734/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:24:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:25:08.618741 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=22 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:25:08.958468 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=66 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:25:18.420724 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfd095-2885-42ad-a940-4178094661e4 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:25:18.421293 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfd095-2885-42ad-a940-4178094661e4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:25:18.421293 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfd095-2885-42ad-a940-4178094661e4 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:25:18.422626 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 737/1471] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:25:18.428533 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605aa10-b581-4321-82ce-ce13cae2f25d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:25:18.428916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605aa10-b581-4321-82ce-ce13cae2f25d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:25:18.429072 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605aa10-b581-4321-82ce-ce13cae2f25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:25:18.493631 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-0605aa10-b581-4321-82ce-ce13cae2f25d None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:25:18.494854 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0605aa10-b581-4321-82ce-ce13cae2f25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:25:18.589501 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 735/1472] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:25:18.678107 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9caef4-239a-4543-ab61-1bf7d075c84f None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:25:18.678391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9caef4-239a-4543-ab61-1bf7d075c84f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:25:18.678723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9caef4-239a-4543-ab61-1bf7d075c84f None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:25:18.681417 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 738/1473] 10.222.0.20 () {54 vars in 966 bytes} [Sat Aug 6 12:25:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:25:18.686098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715f863-a07a-4146-b6ac-fa7a1ce8a316 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:25:18.686381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715f863-a07a-4146-b6ac-fa7a1ce8a316 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:25:18.686659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715f863-a07a-4146-b6ac-fa7a1ce8a316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:25:18.761815 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c715f863-a07a-4146-b6ac-fa7a1ce8a316 None None] MFA Rules not processed for user `7610d6289efb44599d0cee03b93f9938`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:25:18.763015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c715f863-a07a-4146-b6ac-fa7a1ce8a316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:25:18.783386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:25:18.841629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 736/1474] 10.222.0.20 () {58 vars in 1063 bytes} [Sat Aug 6 12:25:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:25:18.850342 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9caef4-239a-4543-ab61-1bf7d075c84f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:25:18.851903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cb9caef4-239a-4543-ab61-1bf7d075c84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:25:18.955836 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9caef4-239a-4543-ab61-1bf7d075c84f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:25:19.037663 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3a008-0783-4470-bd08-4a545460169c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7610d6289efb44599d0cee03b93f9938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:25:19.038279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3a008-0783-4470-bd08-4a545460169c service glance] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:25:19.038558 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3a008-0783-4470-bd08-4a545460169c service glance] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:25:19.038825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3a008-0783-4470-bd08-4a545460169c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:25:19.040466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-17a3a008-0783-4470-bd08-4a545460169c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:25:19.096269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:25:19.151864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3a008-0783-4470-bd08-4a545460169c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:25:19.153317 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3a008-0783-4470-bd08-4a545460169c service glance] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:25:19.171374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 739/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:25:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:25:21.254208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448a378-143e-4b82-86d1-a4b9c5fb7f3e None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:25:21.254710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448a378-143e-4b82-86d1-a4b9c5fb7f3e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:25:21.254710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448a378-143e-4b82-86d1-a4b9c5fb7f3e None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:25:21.256018 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 737/1476] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:25:21.261304 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b1dd8-182a-4363-af02-0e41745947ca None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:25:21.261720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b1dd8-182a-4363-af02-0e41745947ca None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:25:21.261855 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b1dd8-182a-4363-af02-0e41745947ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:25:21.326846 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3b6b1dd8-182a-4363-af02-0e41745947ca None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:25:21.328084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3b6b1dd8-182a-4363-af02-0e41745947ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:25:21.423612 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 740/1477] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:25:21.447241 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448a378-143e-4b82-86d1-a4b9c5fb7f3e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:25:21.448818 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6448a378-143e-4b82-86d1-a4b9c5fb7f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:25:21.570978 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448a378-143e-4b82-86d1-a4b9c5fb7f3e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:25:21.618283 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:25:21.662287 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c3bb6-3f70-4c30-9fe7-262fc64fcadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:25:21.662890 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c3bb6-3f70-4c30-9fe7-262fc64fcadd admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:25:21.663182 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c3bb6-3f70-4c30-9fe7-262fc64fcadd admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:25:21.663471 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c3bb6-3f70-4c30-9fe7-262fc64fcadd admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:25:21.664197 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c3bb6-3f70-4c30-9fe7-262fc64fcadd admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:25:21.665977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: WARNING py.warnings [None req-ba1c3bb6-3f70-4c30-9fe7-262fc64fcadd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 12:25:21.665977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: warnings.warn(msg) Aug 06 12:25:21.665977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: Aug 06 12:25:21.667105 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1c3bb6-3f70-4c30-9fe7-262fc64fcadd admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:25:21.685480 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 738/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Sat Aug 6 12:25:21 2022] GET /identity/v3/projects => generated 2553 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:25:23.680769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059f573-59b5-4f15-8de1-dec8dd4c29e2 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:25:23.681303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059f573-59b5-4f15-8de1-dec8dd4c29e2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:25:23.681303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059f573-59b5-4f15-8de1-dec8dd4c29e2 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:25:23.682679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 741/1479] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:25:23.687980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afde8d7-68f7-43d1-b35b-16373530809b None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:25:23.688461 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afde8d7-68f7-43d1-b35b-16373530809b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:25:23.688572 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afde8d7-68f7-43d1-b35b-16373530809b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:25:23.756489 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-9afde8d7-68f7-43d1-b35b-16373530809b None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:25:23.757706 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9afde8d7-68f7-43d1-b35b-16373530809b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:25:23.851446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 739/1480] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:25:23.894043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7059f573-59b5-4f15-8de1-dec8dd4c29e2 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:25:23.895702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7059f573-59b5-4f15-8de1-dec8dd4c29e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:25:23.905557 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:25:24.039517 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7059f573-59b5-4f15-8de1-dec8dd4c29e2 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:25:24.127411 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:25:24.159596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba82fd5-eac2-46d0-91c3-ab0e673b6d60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:25:24.160200 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba82fd5-eac2-46d0-91c3-ab0e673b6d60 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:25:24.160502 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba82fd5-eac2-46d0-91c3-ab0e673b6d60 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:25:24.160774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba82fd5-eac2-46d0-91c3-ab0e673b6d60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:25:24.162976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0ba82fd5-eac2-46d0-91c3-ab0e673b6d60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:25:24.249639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba82fd5-eac2-46d0-91c3-ab0e673b6d60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:25:24.251140 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba82fd5-eac2-46d0-91c3-ab0e673b6d60 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:25:24.268688 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 742/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:25:26.449455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbac0ad-e3b9-43fe-985e-6ec810d97384 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:25:26.450041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbac0ad-e3b9-43fe-985e-6ec810d97384 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:25:26.450041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbac0ad-e3b9-43fe-985e-6ec810d97384 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:25:26.451390 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 740/1482] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:25:26.456613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f7a42-6574-458e-830f-3a1a6f65eaf4 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:25:26.457021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f7a42-6574-458e-830f-3a1a6f65eaf4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:25:26.457156 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f7a42-6574-458e-830f-3a1a6f65eaf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:25:26.522135 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-508f7a42-6574-458e-830f-3a1a6f65eaf4 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:25:26.523450 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-508f7a42-6574-458e-830f-3a1a6f65eaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:25:26.568762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:25:26.618173 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 743/1483] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:25:26.653278 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbac0ad-e3b9-43fe-985e-6ec810d97384 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:25:26.654777 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fdbac0ad-e3b9-43fe-985e-6ec810d97384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:25:26.678991 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:25:26.756570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbac0ad-e3b9-43fe-985e-6ec810d97384 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:25:26.826179 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77dc31-a1cc-4ca3-9429-cd8ad05772af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:25:26.826779 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77dc31-a1cc-4ca3-9429-cd8ad05772af service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:25:26.827055 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77dc31-a1cc-4ca3-9429-cd8ad05772af service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:25:26.827324 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77dc31-a1cc-4ca3-9429-cd8ad05772af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:25:26.828926 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9d77dc31-a1cc-4ca3-9429-cd8ad05772af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:25:26.901182 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:25:26.936013 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d77dc31-a1cc-4ca3-9429-cd8ad05772af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:25:26.937468 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d77dc31-a1cc-4ca3-9429-cd8ad05772af service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:25:26.954700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 741/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:25:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:25:28.971251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4a288-8dff-4162-8298-8ed9c526eb7c None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:25:28.971830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4a288-8dff-4162-8298-8ed9c526eb7c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:25:28.971830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4a288-8dff-4162-8298-8ed9c526eb7c None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:25:28.973728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 744/1485] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:25:28.978744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af06c19-92ce-4a09-af1c-68a343930546 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:25:28.979134 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af06c19-92ce-4a09-af1c-68a343930546 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:25:28.979290 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af06c19-92ce-4a09-af1c-68a343930546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:25:29.045343 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-4af06c19-92ce-4a09-af1c-68a343930546 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:25:29.046547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4af06c19-92ce-4a09-af1c-68a343930546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:25:29.144332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 742/1486] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:25:29.182929 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4a288-8dff-4162-8298-8ed9c526eb7c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:25:29.184519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5ae4a288-8dff-4162-8298-8ed9c526eb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:25:29.288742 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4a288-8dff-4162-8298-8ed9c526eb7c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:25:29.359617 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:25:29.361478 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f1a096-3206-4645-9083-84c29e53b467 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:25:29.362097 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1a096-3206-4645-9083-84c29e53b467 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:25:29.362826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1a096-3206-4645-9083-84c29e53b467 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:25:29.363155 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1a096-3206-4645-9083-84c29e53b467 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:25:29.365232 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-45f1a096-3206-4645-9083-84c29e53b467 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:25:29.472862 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f1a096-3206-4645-9083-84c29e53b467 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:25:29.474858 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f1a096-3206-4645-9083-84c29e53b467 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:25:29.492669 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 745/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:25:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:25:31.569085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cde49-b8ab-499e-9f95-3d0e57aa0ecb None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:25:31.569617 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cde49-b8ab-499e-9f95-3d0e57aa0ecb None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:25:31.569617 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cde49-b8ab-499e-9f95-3d0e57aa0ecb None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:25:31.571021 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 743/1488] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:25:31.576279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b684fe-fd77-4b3f-bc41-97aac2d4a6f6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:25:31.576729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b684fe-fd77-4b3f-bc41-97aac2d4a6f6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:25:31.576861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b684fe-fd77-4b3f-bc41-97aac2d4a6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:25:31.641416 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-52b684fe-fd77-4b3f-bc41-97aac2d4a6f6 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:25:31.642602 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-52b684fe-fd77-4b3f-bc41-97aac2d4a6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:25:31.696009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:25:31.741501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 746/1489] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:25:31.782322 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1cde49-b8ab-499e-9f95-3d0e57aa0ecb None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:25:31.783848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-da1cde49-b8ab-499e-9f95-3d0e57aa0ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:25:31.819025 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:25:31.885871 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1cde49-b8ab-499e-9f95-3d0e57aa0ecb None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:25:31.956245 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92927f-de2e-4959-8cf1-3ab9e2832c39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:25:31.956906 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92927f-de2e-4959-8cf1-3ab9e2832c39 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:25:31.957186 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92927f-de2e-4959-8cf1-3ab9e2832c39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:25:31.957455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92927f-de2e-4959-8cf1-3ab9e2832c39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:25:31.959722 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ad92927f-de2e-4959-8cf1-3ab9e2832c39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:25:32.041351 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:25:32.070328 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92927f-de2e-4959-8cf1-3ab9e2832c39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:25:32.071796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad92927f-de2e-4959-8cf1-3ab9e2832c39 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:25:32.090059 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 744/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:25:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:25:34.275774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09665729-aeff-44d0-904a-a8cd1ff7eadb None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:25:34.276400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09665729-aeff-44d0-904a-a8cd1ff7eadb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:25:34.276400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09665729-aeff-44d0-904a-a8cd1ff7eadb None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:25:34.277697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 747/1491] 10.222.0.20 () {54 vars in 927 bytes} [Sat Aug 6 12:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:25:34.283022 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae3346-9e0b-4d9e-9831-1294553fefda None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:25:34.283437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae3346-9e0b-4d9e-9831-1294553fefda None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:25:34.283568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae3346-9e0b-4d9e-9831-1294553fefda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:25:34.348099 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-41ae3346-9e0b-4d9e-9831-1294553fefda None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:25:34.349828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-41ae3346-9e0b-4d9e-9831-1294553fefda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:25:34.447766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 745/1492] 10.222.0.20 () {58 vars in 1024 bytes} [Sat Aug 6 12:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:25:34.490053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09665729-aeff-44d0-904a-a8cd1ff7eadb None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:25:34.491553 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-09665729-aeff-44d0-904a-a8cd1ff7eadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:25:34.593366 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09665729-aeff-44d0-904a-a8cd1ff7eadb None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:25:34.667615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25058a77-2bfd-4afe-a928-89c041c05098 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:25:34.668543 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25058a77-2bfd-4afe-a928-89c041c05098 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:25:34.668916 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25058a77-2bfd-4afe-a928-89c041c05098 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:25:34.669284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25058a77-2bfd-4afe-a928-89c041c05098 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:25:34.671671 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-25058a77-2bfd-4afe-a928-89c041c05098 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:25:34.682530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:25:34.784597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25058a77-2bfd-4afe-a928-89c041c05098 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:25:34.786715 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25058a77-2bfd-4afe-a928-89c041c05098 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:25:34.804662 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 748/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:25:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:25:44.445977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=82 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:25:44.802347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=50 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:26:44.469605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef71e14-914d-4da1-be10-6d76758b265e None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:26:44.470192 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef71e14-914d-4da1-be10-6d76758b265e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:26:44.470192 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef71e14-914d-4da1-be10-6d76758b265e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:26:44.545120 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-2ef71e14-914d-4da1-be10-6d76758b265e None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:26:44.546308 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2ef71e14-914d-4da1-be10-6d76758b265e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:26:44.640056 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 746/1494] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:26:44.648837 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a51cf-d6b7-43df-aa3c-ab35708aae06 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:26:44.649305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a51cf-d6b7-43df-aa3c-ab35708aae06 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:26:44.649411 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a51cf-d6b7-43df-aa3c-ab35708aae06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:26:44.725516 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e66a51cf-d6b7-43df-aa3c-ab35708aae06 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:26:44.726642 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e66a51cf-d6b7-43df-aa3c-ab35708aae06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:26:44.821734 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 749/1495] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:26:44.826478 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef71e14-914d-4da1-be10-6d76758b265e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:26:44.827680 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2ef71e14-914d-4da1-be10-6d76758b265e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:26:44.839528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef71e14-914d-4da1-be10-6d76758b265e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:26:44.899435 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:26:44.932061 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7cebac-4805-4552-8a0d-9bb1680ab1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:26:44.932705 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cebac-4805-4552-8a0d-9bb1680ab1ab admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:26:44.932979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cebac-4805-4552-8a0d-9bb1680ab1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:26:44.933246 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cebac-4805-4552-8a0d-9bb1680ab1ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:26:44.933923 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7cebac-4805-4552-8a0d-9bb1680ab1ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:26:44.934234 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7cebac-4805-4552-8a0d-9bb1680ab1ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:26:44.935227 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: WARNING py.warnings [None req-1c7cebac-4805-4552-8a0d-9bb1680ab1ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 06 12:26:44.935227 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: warnings.warn(msg) Aug 06 12:26:44.935227 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: Aug 06 12:26:44.936095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7cebac-4805-4552-8a0d-9bb1680ab1ab admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:26:44.947340 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 747/1496] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:26:44.955803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a51cf-d6b7-43df-aa3c-ab35708aae06 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:26:44.965467 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a51cf-d6b7-43df-aa3c-ab35708aae06 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:26:45.027157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:26:45.060885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3834d4-d8a4-47d7-9e8c-c654fa2e1721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:26:45.061501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3834d4-d8a4-47d7-9e8c-c654fa2e1721 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:26:45.061778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3834d4-d8a4-47d7-9e8c-c654fa2e1721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:26:45.062052 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3834d4-d8a4-47d7-9e8c-c654fa2e1721 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:26:45.062706 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3834d4-d8a4-47d7-9e8c-c654fa2e1721 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:26:45.064465 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: WARNING py.warnings [None req-1e3834d4-d8a4-47d7-9e8c-c654fa2e1721 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 12:26:45.064465 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: warnings.warn(msg) Aug 06 12:26:45.064465 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: Aug 06 12:26:45.065652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3834d4-d8a4-47d7-9e8c-c654fa2e1721 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:26:45.122405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 750/1497] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:26:44 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:26:45.126722 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7cebac-4805-4552-8a0d-9bb1680ab1ab admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:26:45.128239 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-1c7cebac-4805-4552-8a0d-9bb1680ab1ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:26:45.160529 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7cebac-4805-4552-8a0d-9bb1680ab1ab admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:26:45.252961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5ce70-a372-412b-a83e-3db4d82a02c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:26:45.253502 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5ce70-a372-412b-a83e-3db4d82a02c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:26:45.254372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5ce70-a372-412b-a83e-3db4d82a02c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:26:45.254646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5ce70-a372-412b-a83e-3db4d82a02c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:26:45.255295 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5ce70-a372-412b-a83e-3db4d82a02c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:26:45.256328 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: WARNING py.warnings [None req-19f5ce70-a372-412b-a83e-3db4d82a02c8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 12:26:45.256328 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: warnings.warn(msg) Aug 06 12:26:45.256328 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: Aug 06 12:26:45.257526 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5ce70-a372-412b-a83e-3db4d82a02c8 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:26:45.302415 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 748/1498] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:26:45 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:26:45.306524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3834d4-d8a4-47d7-9e8c-c654fa2e1721 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:26:45.308032 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1e3834d4-d8a4-47d7-9e8c-c654fa2e1721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:26:45.370604 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3834d4-d8a4-47d7-9e8c-c654fa2e1721 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:26:45.450811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:26:45.462009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df076d13-2694-4150-8b59-1eb7d523263d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:26:45.462624 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df076d13-2694-4150-8b59-1eb7d523263d admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:26:45.462911 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df076d13-2694-4150-8b59-1eb7d523263d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:26:45.463179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df076d13-2694-4150-8b59-1eb7d523263d admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:26:45.463869 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df076d13-2694-4150-8b59-1eb7d523263d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:26:45.464909 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: WARNING py.warnings [None req-df076d13-2694-4150-8b59-1eb7d523263d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 06 12:26:45.464909 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: warnings.warn(msg) Aug 06 12:26:45.464909 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: Aug 06 12:26:45.465798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df076d13-2694-4150-8b59-1eb7d523263d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:26:45.478438 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 751/1499] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 12:26:45.482506 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5ce70-a372-412b-a83e-3db4d82a02c8 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:26:45.484052 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-19f5ce70-a372-412b-a83e-3db4d82a02c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:26:45.496355 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5ce70-a372-412b-a83e-3db4d82a02c8 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:26:45.520911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:26:45.590646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a65e3-c6f1-4232-b6e2-f75a1340f702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:26:45.591255 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a65e3-c6f1-4232-b6e2-f75a1340f702 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:26:45.591532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a65e3-c6f1-4232-b6e2-f75a1340f702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:26:45.591804 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a65e3-c6f1-4232-b6e2-f75a1340f702 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:26:45.592516 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a65e3-c6f1-4232-b6e2-f75a1340f702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:26:45.593528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: WARNING py.warnings [None req-ba9a65e3-c6f1-4232-b6e2-f75a1340f702 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 06 12:26:45.593528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: warnings.warn(msg) Aug 06 12:26:45.593528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: Aug 06 12:26:45.594404 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a65e3-c6f1-4232-b6e2-f75a1340f702 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:26:45.606292 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 749/1500] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:26:45.610603 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df076d13-2694-4150-8b59-1eb7d523263d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:26:45.612091 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-df076d13-2694-4150-8b59-1eb7d523263d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:26:45.646342 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df076d13-2694-4150-8b59-1eb7d523263d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:26:45.737670 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb649b23-0fac-48f7-a25a-e270942dece5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:26:45.738848 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb649b23-0fac-48f7-a25a-e270942dece5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:26:45.739124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb649b23-0fac-48f7-a25a-e270942dece5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:26:45.739658 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb649b23-0fac-48f7-a25a-e270942dece5 admin admin] PATH_INFO: `/v3/projects/bb5fc0033bef42629d0c7c56185150c7/users/c35dec51248d4789ba62d7b27e485797/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:26:45.761563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb649b23-0fac-48f7-a25a-e270942dece5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb5fc0033bef42629d0c7c56185150c7, user_id=c35dec51248d4789ba62d7b27e485797, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:26:45.762570 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: WARNING py.warnings [None req-bb649b23-0fac-48f7-a25a-e270942dece5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 12:26:45.762570 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: warnings.warn(msg) Aug 06 12:26:45.762570 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: Aug 06 12:26:45.764532 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb649b23-0fac-48f7-a25a-e270942dece5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:26:45.787792 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 752/1501] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:26:45 2022] PUT /identity/v3/projects/bb5fc0033bef42629d0c7c56185150c7/users/c35dec51248d4789ba62d7b27e485797/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:26:45.801022 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbb199-7fa0-40d7-b08a-45a5c2e29c82 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:26:45.801309 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbb199-7fa0-40d7-b08a-45a5c2e29c82 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:26:45.801582 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbb199-7fa0-40d7-b08a-45a5c2e29c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:26:45.857549 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-53dbb199-7fa0-40d7-b08a-45a5c2e29c82 None None] MFA Rules not processed for user `c35dec51248d4789ba62d7b27e485797`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:26:45.858760 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-53dbb199-7fa0-40d7-b08a-45a5c2e29c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:26:45.935117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 750/1502] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:26:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:26:45.949006 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cebd75-7011-4f24-8a1f-3e7c5dd55093 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:26:45.949291 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cebd75-7011-4f24-8a1f-3e7c5dd55093 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:26:45.949563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cebd75-7011-4f24-8a1f-3e7c5dd55093 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:26:45.950905 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 753/1503] 10.222.0.20 () {54 vars in 964 bytes} [Sat Aug 6 12:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:26:45.955823 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e3321-6979-4af5-9a6e-b1ec31e7e7cd None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:26:45.956110 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e3321-6979-4af5-9a6e-b1ec31e7e7cd None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:26:45.956421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e3321-6979-4af5-9a6e-b1ec31e7e7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:26:45.986170 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:26:46.031608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ce9e3321-6979-4af5-9a6e-b1ec31e7e7cd None None] MFA Rules not processed for user `d7a3cbb9c6a34fe2b280873771532e4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:26:46.032849 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ce9e3321-6979-4af5-9a6e-b1ec31e7e7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:26:46.115708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 751/1504] 10.222.0.20 () {58 vars in 1061 bytes} [Sat Aug 6 12:26:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:26:46.123320 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cebd75-7011-4f24-8a1f-3e7c5dd55093 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:26:46.124862 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-35cebd75-7011-4f24-8a1f-3e7c5dd55093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:26:46.173807 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:26:46.227363 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cebd75-7011-4f24-8a1f-3e7c5dd55093 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:26:46.300185 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbab959-77d0-406e-81b3-147be649314e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:26:46.301379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbab959-77d0-406e-81b3-147be649314e service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:26:46.301655 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbab959-77d0-406e-81b3-147be649314e service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:26:46.301930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbab959-77d0-406e-81b3-147be649314e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:26:46.303538 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-adbab959-77d0-406e-81b3-147be649314e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:26:46.399850 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbab959-77d0-406e-81b3-147be649314e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:26:46.401240 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbab959-77d0-406e-81b3-147be649314e service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:26:46.404871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:26:46.420188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 754/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:26:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:26:46.464377 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0d8aa-df78-4d8f-833e-cfb9f5189f4a None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:26:46.464680 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0d8aa-df78-4d8f-833e-cfb9f5189f4a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:26:46.464935 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e0d8aa-df78-4d8f-833e-cfb9f5189f4a None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:26:46.466188 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 752/1506] 10.222.0.20 () {52 vars in 830 bytes} [Sat Aug 6 12:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:26:46.469741 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbab959-77d0-406e-81b3-147be649314e service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:26:46.471123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-adbab959-77d0-406e-81b3-147be649314e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:26:46.569622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbab959-77d0-406e-81b3-147be649314e service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:26:46.662399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d7d58-d60d-4d3d-a574-dbb7c520696e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:26:46.662943 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d7d58-d60d-4d3d-a574-dbb7c520696e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:26:46.663218 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d7d58-d60d-4d3d-a574-dbb7c520696e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:26:46.663487 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d7d58-d60d-4d3d-a574-dbb7c520696e admin admin] PATH_INFO: `/v3/users/c35dec51248d4789ba62d7b27e485797` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:26:46.675820 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d7d58-d60d-4d3d-a574-dbb7c520696e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c35dec51248d4789ba62d7b27e485797)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:26:46.676857 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: WARNING py.warnings [None req-c96d7d58-d60d-4d3d-a574-dbb7c520696e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 12:26:46.676857 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: warnings.warn(msg) Aug 06 12:26:46.676857 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: Aug 06 12:26:46.678061 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d7d58-d60d-4d3d-a574-dbb7c520696e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:26:46.687464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:26:46.742920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c96d7d58-d60d-4d3d-a574-dbb7c520696e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c35dec51248d4789ba62d7b27e485797'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:26:46.765142 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c96d7d58-d60d-4d3d-a574-dbb7c520696e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c35dec51248d4789ba62d7b27e485797'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:26:46.765696 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c96d7d58-d60d-4d3d-a574-dbb7c520696e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c35dec51248d4789ba62d7b27e485797'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:26:46.766121 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c96d7d58-d60d-4d3d-a574-dbb7c520696e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c35dec51248d4789ba62d7b27e485797'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:26:46.775451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c96d7d58-d60d-4d3d-a574-dbb7c520696e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c35dec51248d4789ba62d7b27e485797'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:26:46.794653 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 755/1507] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:26:46 2022] DELETE /identity/v3/users/c35dec51248d4789ba62d7b27e485797 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:26:46.799111 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b4261-e5a3-43fe-9315-515ab6dc31e5 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:26:46.799593 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b4261-e5a3-43fe-9315-515ab6dc31e5 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:26:46.799687 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b4261-e5a3-43fe-9315-515ab6dc31e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:26:46.856818 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-137b4261-e5a3-43fe-9315-515ab6dc31e5 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:26:46.858079 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-137b4261-e5a3-43fe-9315-515ab6dc31e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:26:46.931353 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:26:46.944603 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 753/1508] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:26:46.954060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d7d58-d60d-4d3d-a574-dbb7c520696e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:26:46.955279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c96d7d58-d60d-4d3d-a574-dbb7c520696e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:26:47.054839 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d7d58-d60d-4d3d-a574-dbb7c520696e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:26:47.108739 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:26:47.131268 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf32b2-cd1b-4d4b-ac36-c6632f4719de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:26:47.131893 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf32b2-cd1b-4d4b-ac36-c6632f4719de service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:26:47.132173 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf32b2-cd1b-4d4b-ac36-c6632f4719de service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:26:47.132488 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf32b2-cd1b-4d4b-ac36-c6632f4719de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:26:47.134111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6aaf32b2-cd1b-4d4b-ac36-c6632f4719de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:26:47.245159 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf32b2-cd1b-4d4b-ac36-c6632f4719de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:26:47.246512 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf32b2-cd1b-4d4b-ac36-c6632f4719de service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:26:47.265145 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 756/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:26:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:26:47.584018 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b4261-e5a3-43fe-9315-515ab6dc31e5 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:26:47.585285 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-137b4261-e5a3-43fe-9315-515ab6dc31e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:26:47.596744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b4261-e5a3-43fe-9315-515ab6dc31e5 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:26:47.694872 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9ff6b-bcc5-4302-a716-a6d897c35b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:26:47.695513 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9ff6b-bcc5-4302-a716-a6d897c35b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:26:47.695789 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9ff6b-bcc5-4302-a716-a6d897c35b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:26:47.696059 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9ff6b-bcc5-4302-a716-a6d897c35b6c admin admin] PATH_INFO: `/v3/projects/bb5fc0033bef42629d0c7c56185150c7` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:26:47.705999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9ff6b-bcc5-4302-a716-a6d897c35b6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb5fc0033bef42629d0c7c56185150c7)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:26:47.707706 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: WARNING py.warnings [None req-d8c9ff6b-bcc5-4302-a716-a6d897c35b6c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 12:26:47.707706 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: warnings.warn(msg) Aug 06 12:26:47.707706 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: Aug 06 12:26:47.708934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9ff6b-bcc5-4302-a716-a6d897c35b6c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:26:47.765784 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d8c9ff6b-bcc5-4302-a716-a6d897c35b6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb5fc0033bef42629d0c7c56185150c7', 'request_id': 'req-d8c9ff6b-bcc5-4302-a716-a6d897c35b6c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:26:47.772294 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d8c9ff6b-bcc5-4302-a716-a6d897c35b6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb5fc0033bef42629d0c7c56185150c7', 'request_id': 'req-d8c9ff6b-bcc5-4302-a716-a6d897c35b6c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:26:47.781107 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d8c9ff6b-bcc5-4302-a716-a6d897c35b6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb5fc0033bef42629d0c7c56185150c7', 'request_id': 'req-d8c9ff6b-bcc5-4302-a716-a6d897c35b6c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:26:47.810468 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d8c9ff6b-bcc5-4302-a716-a6d897c35b6c admin admin] The token cache is being invalidate because project bb5fc0033bef42629d0c7c56185150c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:26:47.811053 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d8c9ff6b-bcc5-4302-a716-a6d897c35b6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:26:47.811256 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d8c9ff6b-bcc5-4302-a716-a6d897c35b6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:26:47.812626 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 754/1510] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:26:47 2022] DELETE /identity/v3/projects/bb5fc0033bef42629d0c7c56185150c7 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:26:57.262905 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=62 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:26:57.808467 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:28:12.956241 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98303f4f-da1c-459f-8511-19f548557c04 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:12.956833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98303f4f-da1c-459f-8511-19f548557c04 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:12.956833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98303f4f-da1c-459f-8511-19f548557c04 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:12.958169 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 12:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:28:12.962918 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6f6c9-badf-4c23-9e9d-449eab7b26a6 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:12.963321 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6f6c9-badf-4c23-9e9d-449eab7b26a6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:12.963468 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6f6c9-badf-4c23-9e9d-449eab7b26a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:13.044070 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-dea6f6c9-badf-4c23-9e9d-449eab7b26a6 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:28:13.045305 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-dea6f6c9-badf-4c23-9e9d-449eab7b26a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:28:13.145449 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 12:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:28:13.297042 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98303f4f-da1c-459f-8511-19f548557c04 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:28:13.299177 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-98303f4f-da1c-459f-8511-19f548557c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:28:13.419419 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98303f4f-da1c-459f-8511-19f548557c04 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:28:13.424870 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefe518-5c9d-4801-9818-2b4c34b6e9a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:28:13.425483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefe518-5c9d-4801-9818-2b4c34b6e9a5 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:13.425762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefe518-5c9d-4801-9818-2b4c34b6e9a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:13.426039 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefe518-5c9d-4801-9818-2b4c34b6e9a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:13.434071 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aefe518-5c9d-4801-9818-2b4c34b6e9a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:28:13.435654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aefe518-5c9d-4801-9818-2b4c34b6e9a5 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:28:13.445003 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 758/1513] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:28:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:28:14.278560 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d4f90-f7a0-4f3c-8240-fe253a080eae None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:14.279066 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d4f90-f7a0-4f3c-8240-fe253a080eae None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:14.279066 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d4f90-f7a0-4f3c-8240-fe253a080eae None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:14.280442 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 12:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:28:14.285275 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb31ca-1204-4650-b258-49660d5a5212 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:14.285698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb31ca-1204-4650-b258-49660d5a5212 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:14.285813 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb31ca-1204-4650-b258-49660d5a5212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:14.305495 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0ceb31ca-1204-4650-b258-49660d5a5212 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:28:14.306682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0ceb31ca-1204-4650-b258-49660d5a5212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:28:14.314419 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 12:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:28:14.468184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d4f90-f7a0-4f3c-8240-fe253a080eae None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:28:14.482900 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d4f90-f7a0-4f3c-8240-fe253a080eae None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:28:14.488292 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60a10b-2cc9-4c28-9d35-cdc9f905a0dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:28:14.488931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60a10b-2cc9-4c28-9d35-cdc9f905a0dd service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:14.489218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60a10b-2cc9-4c28-9d35-cdc9f905a0dd service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:14.489495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60a10b-2cc9-4c28-9d35-cdc9f905a0dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:14.498362 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60a10b-2cc9-4c28-9d35-cdc9f905a0dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:28:14.499945 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60a10b-2cc9-4c28-9d35-cdc9f905a0dd service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:28:14.509242 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 757/1516] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:28:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:28:14.922632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de907dc-a259-49c8-af33-9e59c778a760 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:14.922838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de907dc-a259-49c8-af33-9e59c778a760 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:14.923098 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de907dc-a259-49c8-af33-9e59c778a760 None None] PATH_INFO: `/v3` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:14.924412 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 760/1517] 10.222.0.27 () {54 vars in 925 bytes} [Sat Aug 6 12:28:14 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:28:14.930328 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a3afd-30b6-4e5b-9384-f606b4f6995c None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:14.930612 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a3afd-30b6-4e5b-9384-f606b4f6995c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:14.930888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a3afd-30b6-4e5b-9384-f606b4f6995c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:14.989990 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-1b4a3afd-30b6-4e5b-9384-f606b4f6995c None None] MFA Rules not processed for user `e2244655ba304b9c980b5e68772eefc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:28:14.991453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-1b4a3afd-30b6-4e5b-9384-f606b4f6995c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:28:15.008438 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:28:15.062257 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 758/1518] 10.222.0.27 () {58 vars in 1015 bytes} [Sat Aug 6 12:28:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:28:24.056780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb0373-94f4-4bfc-b2ca-7efb572636e5 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:24.057255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb0373-94f4-4bfc-b2ca-7efb572636e5 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:24.057255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debb0373-94f4-4bfc-b2ca-7efb572636e5 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:24.058586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 761/1519] 10.222.0.20 () {54 vars in 961 bytes} [Sat Aug 6 12:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:28:24.063738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fb8e3-0414-4d0a-8aea-98c91e03b20d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:24.064183 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fb8e3-0414-4d0a-8aea-98c91e03b20d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:24.064290 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fb8e3-0414-4d0a-8aea-98c91e03b20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:24.083680 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-961fb8e3-0414-4d0a-8aea-98c91e03b20d None None] MFA Rules not processed for user `e2244655ba304b9c980b5e68772eefc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:28:24.084902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-961fb8e3-0414-4d0a-8aea-98c91e03b20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:28:24.093247 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 759/1520] 10.222.0.20 () {58 vars in 1058 bytes} [Sat Aug 6 12:28:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:28:24.100454 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb0373-94f4-4bfc-b2ca-7efb572636e5 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:28:24.113743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debb0373-94f4-4bfc-b2ca-7efb572636e5 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:28:24.119925 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52974c-6c76-4ba1-8f77-cd0ef57479bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2244655ba304b9c980b5e68772eefc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:28:24.120564 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52974c-6c76-4ba1-8f77-cd0ef57479bd service placement] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:24.120841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52974c-6c76-4ba1-8f77-cd0ef57479bd service placement] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:24.121110 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52974c-6c76-4ba1-8f77-cd0ef57479bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:24.129168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52974c-6c76-4ba1-8f77-cd0ef57479bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:28:24.130717 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52974c-6c76-4ba1-8f77-cd0ef57479bd service placement] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:28:24.141183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 762/1521] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:28:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:28:25.744327 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd904a32-a1cd-4660-b0b0-352049144382 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:25.744931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd904a32-a1cd-4660-b0b0-352049144382 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:25.744931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd904a32-a1cd-4660-b0b0-352049144382 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:25.746294 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 12:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:28:25.751151 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4daaea-fa21-4815-8166-c864d512c5d2 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:25.751574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4daaea-fa21-4815-8166-c864d512c5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:25.751686 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4daaea-fa21-4815-8166-c864d512c5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:25.770531 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5b4daaea-fa21-4815-8166-c864d512c5d2 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:28:25.771983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5b4daaea-fa21-4815-8166-c864d512c5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:28:25.779884 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 12:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:28:25.936696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd904a32-a1cd-4660-b0b0-352049144382 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:28:25.951294 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd904a32-a1cd-4660-b0b0-352049144382 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:28:25.956678 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f92936-ef04-4f8c-8810-031cdc7c53f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:28:25.957279 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f92936-ef04-4f8c-8810-031cdc7c53f7 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:25.957559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f92936-ef04-4f8c-8810-031cdc7c53f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:25.957827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f92936-ef04-4f8c-8810-031cdc7c53f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:25.965821 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f92936-ef04-4f8c-8810-031cdc7c53f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:28:25.967389 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f92936-ef04-4f8c-8810-031cdc7c53f7 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:28:25.977016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 761/1524] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:28:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:28:26.800350 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acb788-393b-4f20-88b7-2b11ae66ad12 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:26.800923 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acb788-393b-4f20-88b7-2b11ae66ad12 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:26.800923 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acb788-393b-4f20-88b7-2b11ae66ad12 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:26.802249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat Aug 6 12:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:28:26.807025 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267c26f-e8d3-4f27-99d4-521f9f300f12 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:26.807456 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267c26f-e8d3-4f27-99d4-521f9f300f12 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:26.807568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267c26f-e8d3-4f27-99d4-521f9f300f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:26.827176 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-6267c26f-e8d3-4f27-99d4-521f9f300f12 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:28:26.828393 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6267c26f-e8d3-4f27-99d4-521f9f300f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:28:26.836825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat Aug 6 12:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:28:26.989266 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13acb788-393b-4f20-88b7-2b11ae66ad12 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:28:27.004478 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13acb788-393b-4f20-88b7-2b11ae66ad12 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:28:27.010430 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0560aa5-b7f8-4987-a967-e2b839e3bd7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:28:27.011034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0560aa5-b7f8-4987-a967-e2b839e3bd7c service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:27.011346 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0560aa5-b7f8-4987-a967-e2b839e3bd7c service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:27.011620 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0560aa5-b7f8-4987-a967-e2b839e3bd7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:27.019560 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0560aa5-b7f8-4987-a967-e2b839e3bd7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:28:27.021143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0560aa5-b7f8-4987-a967-e2b839e3bd7c service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:28:27.030490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 765/1527] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:28:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:28:30.717077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6cff12-19ac-4581-8d59-687fd70a7471 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:30.717661 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6cff12-19ac-4581-8d59-687fd70a7471 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:30.717661 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6cff12-19ac-4581-8d59-687fd70a7471 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:30.718970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 12:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:28:30.724161 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724df070-630f-4e04-95d3-608da60d5be5 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:30.724595 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724df070-630f-4e04-95d3-608da60d5be5 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:30.724757 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724df070-630f-4e04-95d3-608da60d5be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:30.744560 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-724df070-630f-4e04-95d3-608da60d5be5 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:28:30.745761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-724df070-630f-4e04-95d3-608da60d5be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:28:30.753542 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 12:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:28:30.825336 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6cff12-19ac-4581-8d59-687fd70a7471 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:28:30.827026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2e6cff12-19ac-4581-8d59-687fd70a7471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:28:30.902818 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6cff12-19ac-4581-8d59-687fd70a7471 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:28:30.906240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b28bb5a-c38b-46fa-94e3-34b62d439352 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:28:30.906841 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28bb5a-c38b-46fa-94e3-34b62d439352 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:30.907120 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28bb5a-c38b-46fa-94e3-34b62d439352 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:30.907392 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28bb5a-c38b-46fa-94e3-34b62d439352 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:30.916713 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b28bb5a-c38b-46fa-94e3-34b62d439352 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:28:30.918132 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b28bb5a-c38b-46fa-94e3-34b62d439352 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:28:30.927313 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 764/1530] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:28:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:28:32.912765 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63c0ec-dbb2-4227-95d7-b16016e67e8f None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:32.913838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63c0ec-dbb2-4227-95d7-b16016e67e8f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:32.913838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63c0ec-dbb2-4227-95d7-b16016e67e8f None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:32.915163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 12:28:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:28:32.920240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd07136-3cc4-440b-a2bb-362afa1bdfe3 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:32.920697 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd07136-3cc4-440b-a2bb-362afa1bdfe3 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:32.920838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd07136-3cc4-440b-a2bb-362afa1bdfe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:32.940120 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-9bd07136-3cc4-440b-a2bb-362afa1bdfe3 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:28:32.941350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9bd07136-3cc4-440b-a2bb-362afa1bdfe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:28:32.949151 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 12:28:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:28:32.986649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63c0ec-dbb2-4227-95d7-b16016e67e8f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:28:32.988386 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ff63c0ec-dbb2-4227-95d7-b16016e67e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:28:33.001747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63c0ec-dbb2-4227-95d7-b16016e67e8f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:28:33.005572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1e27e-b8f3-4675-a172-2693a24484ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:28:33.006313 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1e27e-b8f3-4675-a172-2693a24484ed service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:33.006622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1e27e-b8f3-4675-a172-2693a24484ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:33.006945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1e27e-b8f3-4675-a172-2693a24484ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:33.016883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff1e27e-b8f3-4675-a172-2693a24484ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:28:33.018496 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff1e27e-b8f3-4675-a172-2693a24484ed service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:28:33.029308 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 768/1533] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:28:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:28:36.520048 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c091f42-a656-471c-9018-fc7154c877c0 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:36.520660 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c091f42-a656-471c-9018-fc7154c877c0 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:36.520660 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c091f42-a656-471c-9018-fc7154c877c0 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:36.521977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 12:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:28:36.529246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45092bb4-6495-41e3-8929-f591a537751a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:36.529704 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45092bb4-6495-41e3-8929-f591a537751a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:36.529800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45092bb4-6495-41e3-8929-f591a537751a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:36.549644 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-45092bb4-6495-41e3-8929-f591a537751a None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:28:36.551036 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-45092bb4-6495-41e3-8929-f591a537751a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:28:36.558885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 12:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:28:36.869332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c091f42-a656-471c-9018-fc7154c877c0 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:28:36.871023 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-1c091f42-a656-471c-9018-fc7154c877c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:28:36.934818 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:28:36.948778 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c091f42-a656-471c-9018-fc7154c877c0 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:28:36.952357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c8f54-7121-4246-8100-6e6ed63f43c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:28:36.952972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c8f54-7121-4246-8100-6e6ed63f43c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:36.953254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c8f54-7121-4246-8100-6e6ed63f43c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:36.953525 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c8f54-7121-4246-8100-6e6ed63f43c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:36.962265 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c8f54-7121-4246-8100-6e6ed63f43c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:28:36.963688 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c8f54-7121-4246-8100-6e6ed63f43c0 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:28:36.972850 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 767/1536] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:28:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:28:39.009413 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e0c07-39f7-4c83-8a2f-71277d011428 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:39.009821 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e0c07-39f7-4c83-8a2f-71277d011428 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:39.009967 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e0c07-39f7-4c83-8a2f-71277d011428 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:39.011342 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sat Aug 6 12:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:28:39.016535 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67ab48-6e12-4502-9519-c5d03647ef05 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:39.016944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67ab48-6e12-4502-9519-c5d03647ef05 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:39.017085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67ab48-6e12-4502-9519-c5d03647ef05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:39.036403 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-5d67ab48-6e12-4502-9519-c5d03647ef05 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:28:39.037588 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5d67ab48-6e12-4502-9519-c5d03647ef05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:28:39.045333 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Aug 6 12:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:28:39.349184 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e0c07-39f7-4c83-8a2f-71277d011428 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:28:39.351419 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-696e0c07-39f7-4c83-8a2f-71277d011428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:28:39.358997 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=84 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:28:39.364885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e0c07-39f7-4c83-8a2f-71277d011428 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:28:39.369270 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c095a042-5268-43a1-aac6-f3cd35f2b508 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:28:39.369872 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095a042-5268-43a1-aac6-f3cd35f2b508 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:28:39.370155 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095a042-5268-43a1-aac6-f3cd35f2b508 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:28:39.370428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095a042-5268-43a1-aac6-f3cd35f2b508 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:28:39.379253 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c095a042-5268-43a1-aac6-f3cd35f2b508 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:28:39.380707 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c095a042-5268-43a1-aac6-f3cd35f2b508 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:28:39.390508 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 771/1539] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:28:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:28:49.031571 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=11 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:28:49.386382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:01.795299 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996481a-ec42-4b06-afaf-b20e1e414747 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:01.796522 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996481a-ec42-4b06-afaf-b20e1e414747 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:01.796522 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996481a-ec42-4b06-afaf-b20e1e414747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:01.821404 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-f996481a-ec42-4b06-afaf-b20e1e414747 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:01.822627 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f996481a-ec42-4b06-afaf-b20e1e414747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:01.838556 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 769/1540] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:01.847158 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f41af7-2cd1-4639-ac63-13e544e422c7 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:01.847800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f41af7-2cd1-4639-ac63-13e544e422c7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:01.847800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f41af7-2cd1-4639-ac63-13e544e422c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:01.882064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-53f41af7-2cd1-4639-ac63-13e544e422c7 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:01.883305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-53f41af7-2cd1-4639-ac63-13e544e422c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:01.901771 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 772/1541] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:01.906605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996481a-ec42-4b06-afaf-b20e1e414747 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:01.917243 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c753ca-aeb1-490b-a5bc-7004240eddf8 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:01.917594 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c753ca-aeb1-490b-a5bc-7004240eddf8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:01.917812 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c753ca-aeb1-490b-a5bc-7004240eddf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:01.926960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996481a-ec42-4b06-afaf-b20e1e414747 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:01.949048 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5707f-19d8-444b-a753-c17829d17d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:01.949656 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5707f-19d8-444b-a753-c17829d17d6f admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:01.949935 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5707f-19d8-444b-a753-c17829d17d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:01.950208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5707f-19d8-444b-a753-c17829d17d6f admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:01.950902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5707f-19d8-444b-a753-c17829d17d6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:30:01.951203 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5707f-19d8-444b-a753-c17829d17d6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:01.952860 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5707f-19d8-444b-a753-c17829d17d6f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:01.953794 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-00c753ca-aeb1-490b-a5bc-7004240eddf8 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:01.955026 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-00c753ca-aeb1-490b-a5bc-7004240eddf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:01.974930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 773/1542] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:01.976406 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 770/1543] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:30:01.991739 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5707f-19d8-444b-a753-c17829d17d6f admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:01.992218 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580aa1a-3789-4f12-b564-2b1fc2ffe2c6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:01.992388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580aa1a-3789-4f12-b564-2b1fc2ffe2c6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:01.992674 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580aa1a-3789-4f12-b564-2b1fc2ffe2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.014714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5707f-19d8-444b-a753-c17829d17d6f admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:02.035958 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d580aa1a-3789-4f12-b564-2b1fc2ffe2c6 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:02.037844 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d580aa1a-3789-4f12-b564-2b1fc2ffe2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:02.046285 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dafe6a5-5adc-4256-88a5-4d409d427ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:02.046285 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafe6a5-5adc-4256-88a5-4d409d427ec1 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.046285 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafe6a5-5adc-4256-88a5-4d409d427ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.046285 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafe6a5-5adc-4256-88a5-4d409d427ec1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.046961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dafe6a5-5adc-4256-88a5-4d409d427ec1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:02.047993 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: WARNING py.warnings [None req-7dafe6a5-5adc-4256-88a5-4d409d427ec1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 12:30:02.047993 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: warnings.warn(msg) Aug 06 12:30:02.047993 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: Aug 06 12:30:02.049336 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dafe6a5-5adc-4256-88a5-4d409d427ec1 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:02.057897 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 774/1544] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:02.059425 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a927a20-269c-41b6-ada3-bdcc100e587b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.059712 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a927a20-269c-41b6-ada3-bdcc100e587b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.059984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a927a20-269c-41b6-ada3-bdcc100e587b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.096979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6a927a20-269c-41b6-ada3-bdcc100e587b None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:02.098609 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6a927a20-269c-41b6-ada3-bdcc100e587b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:02.118838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 775/1545] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:02.119719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a927a20-269c-41b6-ada3-bdcc100e587b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:02.136974 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a927a20-269c-41b6-ada3-bdcc100e587b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:02.155302 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87d7a5-881e-4515-8445-97426e86a884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:02.155302 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87d7a5-881e-4515-8445-97426e86a884 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.155302 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87d7a5-881e-4515-8445-97426e86a884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.155302 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87d7a5-881e-4515-8445-97426e86a884 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.155788 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 771/1546] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:01 2022] POST /identity/v3/projects => generated 381 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 12:30:02.155876 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87d7a5-881e-4515-8445-97426e86a884 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:30:02.155876 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87d7a5-881e-4515-8445-97426e86a884 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:02.156033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe983c53-8215-43ae-8699-5217333e1167 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.156033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe983c53-8215-43ae-8699-5217333e1167 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.156179 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe983c53-8215-43ae-8699-5217333e1167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.156251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: WARNING py.warnings [None req-6f87d7a5-881e-4515-8445-97426e86a884 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 06 12:30:02.156251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: warnings.warn(msg) Aug 06 12:30:02.156251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: Aug 06 12:30:02.157739 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87d7a5-881e-4515-8445-97426e86a884 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:02.176168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 776/1547] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:30:02.178151 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87d7a5-881e-4515-8445-97426e86a884 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:02.194661 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87d7a5-881e-4515-8445-97426e86a884 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:02.201657 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-fe983c53-8215-43ae-8699-5217333e1167 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:02.202891 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fe983c53-8215-43ae-8699-5217333e1167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:02.295272 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f320a2-993b-4b08-a5e8-e436066999e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:02.295928 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f320a2-993b-4b08-a5e8-e436066999e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.296211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f320a2-993b-4b08-a5e8-e436066999e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.296583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f320a2-993b-4b08-a5e8-e436066999e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.297259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f320a2-993b-4b08-a5e8-e436066999e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:02.298251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: WARNING py.warnings [None req-70f320a2-993b-4b08-a5e8-e436066999e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 12:30:02.298251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: warnings.warn(msg) Aug 06 12:30:02.298251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: Aug 06 12:30:02.299612 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f320a2-993b-4b08-a5e8-e436066999e4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:02.324006 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 772/1548] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:02.325394 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe983c53-8215-43ae-8699-5217333e1167 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:02.340373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe983c53-8215-43ae-8699-5217333e1167 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:02.351200 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 777/1549] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:02 2022] POST /identity/v3/users => generated 291 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:02.352866 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46fc72-b23b-4045-8bba-33c0715ed8d7 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.353401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46fc72-b23b-4045-8bba-33c0715ed8d7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.353897 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b46fc72-b23b-4045-8bba-33c0715ed8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.359752 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf3f17-aa37-42a1-808f-4f9a47ff008e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:02.361151 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf3f17-aa37-42a1-808f-4f9a47ff008e admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.362256 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf3f17-aa37-42a1-808f-4f9a47ff008e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.363288 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf3f17-aa37-42a1-808f-4f9a47ff008e admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.364230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf3f17-aa37-42a1-808f-4f9a47ff008e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:02.369694 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf3f17-aa37-42a1-808f-4f9a47ff008e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:02.394214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1b46fc72-b23b-4045-8bba-33c0715ed8d7 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:02.395433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1b46fc72-b23b-4045-8bba-33c0715ed8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:02.413871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 778/1550] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:02.417365 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 773/1551] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:02 2022] POST /identity/v3/projects => generated 381 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:02.418219 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf3f17-aa37-42a1-808f-4f9a47ff008e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:02.418876 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24b98e-2c25-4af9-8901-33c0b1e6690a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.419158 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24b98e-2c25-4af9-8901-33c0b1e6690a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.419438 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24b98e-2c25-4af9-8901-33c0b1e6690a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.458640 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ca24b98e-2c25-4af9-8901-33c0b1e6690a None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:02.459998 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ca24b98e-2c25-4af9-8901-33c0b1e6690a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:02.500558 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf3f17-aa37-42a1-808f-4f9a47ff008e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:02.506062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e66318-f34b-4564-94dd-6652efaebe99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:02.506945 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e66318-f34b-4564-94dd-6652efaebe99 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.507476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e66318-f34b-4564-94dd-6652efaebe99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.507921 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e66318-f34b-4564-94dd-6652efaebe99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.508652 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e66318-f34b-4564-94dd-6652efaebe99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:30:02.508961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e66318-f34b-4564-94dd-6652efaebe99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:02.510569 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e66318-f34b-4564-94dd-6652efaebe99 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:02.523152 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 774/1552] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:30:02.524129 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e66318-f34b-4564-94dd-6652efaebe99 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:02.528760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 779/1553] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:02.530796 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a1aee-12c2-4aae-9694-b46ae625692a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.531087 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a1aee-12c2-4aae-9694-b46ae625692a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.531363 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968a1aee-12c2-4aae-9694-b46ae625692a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.534002 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e66318-f34b-4564-94dd-6652efaebe99 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:02.539289 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029258d-4488-484e-8826-70aedaee28b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:02.539893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029258d-4488-484e-8826-70aedaee28b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.540174 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029258d-4488-484e-8826-70aedaee28b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.540498 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029258d-4488-484e-8826-70aedaee28b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.541193 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f029258d-4488-484e-8826-70aedaee28b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:02.542786 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f029258d-4488-484e-8826-70aedaee28b7 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:02.552397 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-968a1aee-12c2-4aae-9694-b46ae625692a None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:02.553951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 775/1554] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:02.554088 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-968a1aee-12c2-4aae-9694-b46ae625692a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:02.554861 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029258d-4488-484e-8826-70aedaee28b7 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:02.562268 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 780/1555] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:02.563437 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bd09b-4da4-4bef-9738-f9ceb26cefda None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.563716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bd09b-4da4-4bef-9738-f9ceb26cefda None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.563995 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bd09b-4da4-4bef-9738-f9ceb26cefda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.564670 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029258d-4488-484e-8826-70aedaee28b7 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:02.569401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e24e33-029b-4ef9-896f-846c189bdef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:02.569993 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e24e33-029b-4ef9-896f-846c189bdef5 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.570272 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e24e33-029b-4ef9-896f-846c189bdef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.570542 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e24e33-029b-4ef9-896f-846c189bdef5 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.571196 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e24e33-029b-4ef9-896f-846c189bdef5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:02.573186 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e24e33-029b-4ef9-896f-846c189bdef5 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:02.582671 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-446bd09b-4da4-4bef-9738-f9ceb26cefda None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:02.583215 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:02.583715 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-446bd09b-4da4-4bef-9738-f9ceb26cefda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:02.595702 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 776/1556] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:02 2022] POST /identity/v3/users => generated 291 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:02.597176 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9a980-2ee9-4aba-b094-166130c1baa4 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.597464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9a980-2ee9-4aba-b094-166130c1baa4 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.597740 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9a980-2ee9-4aba-b094-166130c1baa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.600250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 781/1557] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:02.601596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87756d0-cda2-4587-a5fe-e2cb12f6259c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.601878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87756d0-cda2-4587-a5fe-e2cb12f6259c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.602150 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87756d0-cda2-4587-a5fe-e2cb12f6259c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.617987 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-eed9a980-2ee9-4aba-b094-166130c1baa4 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:02.619177 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-eed9a980-2ee9-4aba-b094-166130c1baa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:02.621207 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e87756d0-cda2-4587-a5fe-e2cb12f6259c None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:02.622346 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e87756d0-cda2-4587-a5fe-e2cb12f6259c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:02.627318 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 777/1558] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:02.628596 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1adea0-577b-4a54-937a-3911a831a0ed None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.628880 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1adea0-577b-4a54-937a-3911a831a0ed None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.629154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1adea0-577b-4a54-937a-3911a831a0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.630911 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 782/1559] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:02.631620 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87756d0-cda2-4587-a5fe-e2cb12f6259c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:02.642081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87756d0-cda2-4587-a5fe-e2cb12f6259c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:02.646882 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fcdedb-5e4c-4ef0-a3bb-e5e7f5d5389f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:02.647492 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fcdedb-5e4c-4ef0-a3bb-e5e7f5d5389f admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.647768 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fcdedb-5e4c-4ef0-a3bb-e5e7f5d5389f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.648037 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fcdedb-5e4c-4ef0-a3bb-e5e7f5d5389f admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.648728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fcdedb-5e4c-4ef0-a3bb-e5e7f5d5389f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:02.650629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fcdedb-5e4c-4ef0-a3bb-e5e7f5d5389f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:02.653138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-bf1adea0-577b-4a54-937a-3911a831a0ed None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:02.654327 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-bf1adea0-577b-4a54-937a-3911a831a0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:02.662639 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 778/1560] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:02.663473 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1adea0-577b-4a54-937a-3911a831a0ed None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:02.673808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1adea0-577b-4a54-937a-3911a831a0ed None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:02.679455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c55eab-386a-4f08-a494-d6dc2cd9d7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:02.680176 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c55eab-386a-4f08-a494-d6dc2cd9d7e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.680490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c55eab-386a-4f08-a494-d6dc2cd9d7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.680758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c55eab-386a-4f08-a494-d6dc2cd9d7e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.681438 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c55eab-386a-4f08-a494-d6dc2cd9d7e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:02.685410 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 783/1561] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:02 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:02.685567 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c55eab-386a-4f08-a494-d6dc2cd9d7e9 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:02.685666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fcdedb-5e4c-4ef0-a3bb-e5e7f5d5389f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:02.689867 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:02.694319 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 779/1562] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:02.695131 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c55eab-386a-4f08-a494-d6dc2cd9d7e9 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:02.704632 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c55eab-386a-4f08-a494-d6dc2cd9d7e9 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:02.749915 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fcdedb-5e4c-4ef0-a3bb-e5e7f5d5389f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:02.754513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e25dd-7c97-4187-9dec-d2e8802490cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:02.755115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e25dd-7c97-4187-9dec-d2e8802490cf admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.755392 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e25dd-7c97-4187-9dec-d2e8802490cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.755688 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e25dd-7c97-4187-9dec-d2e8802490cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.756405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e25dd-7c97-4187-9dec-d2e8802490cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:30:02.756711 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e25dd-7c97-4187-9dec-d2e8802490cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:02.758299 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e25dd-7c97-4187-9dec-d2e8802490cf admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:02.760168 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87fbb34-f97b-4fbe-9176-92f0829451c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:02.760798 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fbb34-f97b-4fbe-9176-92f0829451c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.761077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fbb34-f97b-4fbe-9176-92f0829451c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.761352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fbb34-f97b-4fbe-9176-92f0829451c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.762040 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87fbb34-f97b-4fbe-9176-92f0829451c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:02.763623 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87fbb34-f97b-4fbe-9176-92f0829451c8 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:02.769890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 784/1563] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:30:02.770958 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b8e93-f21a-498e-a8e2-f304f61df810 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.771758 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b8e93-f21a-498e-a8e2-f304f61df810 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.772028 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b8e93-f21a-498e-a8e2-f304f61df810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.776987 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 780/1564] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:02.776987 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736cd13-507b-478c-9ac0-0b35d0f86e31 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.776987 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736cd13-507b-478c-9ac0-0b35d0f86e31 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.776987 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736cd13-507b-478c-9ac0-0b35d0f86e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.793603 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-303b8e93-f21a-498e-a8e2-f304f61df810 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:02.794791 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-303b8e93-f21a-498e-a8e2-f304f61df810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:02.795934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-5736cd13-507b-478c-9ac0-0b35d0f86e31 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:02.797151 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5736cd13-507b-478c-9ac0-0b35d0f86e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:02.803132 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 785/1565] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:02.804231 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583e4ee-fb8c-41b9-959e-44f87edbfbb0 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.804535 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583e4ee-fb8c-41b9-959e-44f87edbfbb0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.804807 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583e4ee-fb8c-41b9-959e-44f87edbfbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.808153 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 781/1566] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:02.812625 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736cd13-507b-478c-9ac0-0b35d0f86e31 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:02.822190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736cd13-507b-478c-9ac0-0b35d0f86e31 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:02.823750 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f583e4ee-fb8c-41b9-959e-44f87edbfbb0 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:02.825188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f583e4ee-fb8c-41b9-959e-44f87edbfbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:02.826882 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db513df3-286b-4e00-b4c8-9a5cc3eaa151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:02.827594 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db513df3-286b-4e00-b4c8-9a5cc3eaa151 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.827871 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db513df3-286b-4e00-b4c8-9a5cc3eaa151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.828142 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db513df3-286b-4e00-b4c8-9a5cc3eaa151 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.828899 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db513df3-286b-4e00-b4c8-9a5cc3eaa151 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:30:02.829189 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db513df3-286b-4e00-b4c8-9a5cc3eaa151 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:02.830826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db513df3-286b-4e00-b4c8-9a5cc3eaa151 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:02.833182 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 786/1567] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:02.834438 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583e4ee-fb8c-41b9-959e-44f87edbfbb0 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:02.843544 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 782/1568] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:30:02.843690 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583e4ee-fb8c-41b9-959e-44f87edbfbb0 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:02.844656 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db513df3-286b-4e00-b4c8-9a5cc3eaa151 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:02.848057 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d077d9b9-eca0-4622-8575-9716858014b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:02.848699 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d077d9b9-eca0-4622-8575-9716858014b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.848982 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d077d9b9-eca0-4622-8575-9716858014b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.849247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d077d9b9-eca0-4622-8575-9716858014b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.849972 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d077d9b9-eca0-4622-8575-9716858014b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:02.852146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d077d9b9-eca0-4622-8575-9716858014b9 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:02.855530 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db513df3-286b-4e00-b4c8-9a5cc3eaa151 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:02.860135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8e427-392a-43a9-9449-5bce9dfdcdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:02.860823 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8e427-392a-43a9-9449-5bce9dfdcdb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.861103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8e427-392a-43a9-9449-5bce9dfdcdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.861378 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8e427-392a-43a9-9449-5bce9dfdcdb0 admin admin] PATH_INFO: `/v3/projects/1b036d583edb46fcb6a545db4952e6b3/users/6931920e71fc4cd69e0d654bf2b8badc/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.874571 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 787/1569] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:02 2022] POST /identity/v3/users => generated 295 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:02.875537 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d077d9b9-eca0-4622-8575-9716858014b9 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:02.877485 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8e427-392a-43a9-9449-5bce9dfdcdb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b036d583edb46fcb6a545db4952e6b3, user_id=6931920e71fc4cd69e0d654bf2b8badc, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:02.878602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: WARNING py.warnings [None req-dfb8e427-392a-43a9-9449-5bce9dfdcdb0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 12:30:02.878602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: warnings.warn(msg) Aug 06 12:30:02.878602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: Aug 06 12:30:02.880553 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8e427-392a-43a9-9449-5bce9dfdcdb0 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:02.885765 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d077d9b9-eca0-4622-8575-9716858014b9 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:02.891056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca1309-c2b0-4703-bd42-eeced0c10816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:02.891660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca1309-c2b0-4703-bd42-eeced0c10816 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.891939 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca1309-c2b0-4703-bd42-eeced0c10816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.892213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca1309-c2b0-4703-bd42-eeced0c10816 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.892929 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca1309-c2b0-4703-bd42-eeced0c10816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:02.894498 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ca1309-c2b0-4703-bd42-eeced0c10816 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:02.895448 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 783/1570] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:02 2022] PUT /identity/v3/projects/1b036d583edb46fcb6a545db4952e6b3/users/6931920e71fc4cd69e0d654bf2b8badc/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:02.896755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8e427-392a-43a9-9449-5bce9dfdcdb0 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:02.910453 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 788/1571] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:02.911346 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca1309-c2b0-4703-bd42-eeced0c10816 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:02.921010 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca1309-c2b0-4703-bd42-eeced0c10816 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:02.961906 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8e427-392a-43a9-9449-5bce9dfdcdb0 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:02.966679 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebbdcb-c268-4743-9ed6-f991c81c2aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:02.967318 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebbdcb-c268-4743-9ed6-f991c81c2aa1 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.967546 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebbdcb-c268-4743-9ed6-f991c81c2aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.967819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebbdcb-c268-4743-9ed6-f991c81c2aa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.968496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ebbdcb-c268-4743-9ed6-f991c81c2aa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:02.970404 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ebbdcb-c268-4743-9ed6-f991c81c2aa1 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:02.978410 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c9935-be99-4d32-a583-05551a941337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:02.979025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c9935-be99-4d32-a583-05551a941337 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:02.979596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c9935-be99-4d32-a583-05551a941337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:02.979596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c9935-be99-4d32-a583-05551a941337 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:02.980277 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c9935-be99-4d32-a583-05551a941337 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:30:02.980610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c9935-be99-4d32-a583-05551a941337 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:02.982497 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c9935-be99-4d32-a583-05551a941337 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:02.998334 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 784/1572] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:02 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:02.999172 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebbdcb-c268-4743-9ed6-f991c81c2aa1 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.000845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 789/1573] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:30:03.001760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c9935-be99-4d32-a583-05551a941337 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.013426 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:03.015935 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c9935-be99-4d32-a583-05551a941337 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.060684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:03.068876 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebbdcb-c268-4743-9ed6-f991c81c2aa1 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.073808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cebc46-174a-479a-acc6-d5af4781cdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.074426 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cebc46-174a-479a-acc6-d5af4781cdf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.074704 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cebc46-174a-479a-acc6-d5af4781cdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.074982 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cebc46-174a-479a-acc6-d5af4781cdf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.075115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021835c-7068-4c29-a20c-2a30129e4a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.075665 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021835c-7068-4c29-a20c-2a30129e4a1d admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.075782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cebc46-174a-479a-acc6-d5af4781cdf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:30:03.075965 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021835c-7068-4c29-a20c-2a30129e4a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.076184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cebc46-174a-479a-acc6-d5af4781cdf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.076279 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021835c-7068-4c29-a20c-2a30129e4a1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.077301 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021835c-7068-4c29-a20c-2a30129e4a1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:30:03.077604 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cebc46-174a-479a-acc6-d5af4781cdf8 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.077720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021835c-7068-4c29-a20c-2a30129e4a1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.079164 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021835c-7068-4c29-a20c-2a30129e4a1d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.089109 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 785/1574] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:30:03.090040 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cebc46-174a-479a-acc6-d5af4781cdf8 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.105216 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 790/1575] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:30:03.105216 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021835c-7068-4c29-a20c-2a30129e4a1d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.105488 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cebc46-174a-479a-acc6-d5af4781cdf8 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.105488 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cbcd5-2cbe-42a1-be98-6b2e726b98de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.105488 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cbcd5-2cbe-42a1-be98-6b2e726b98de admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.105488 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cbcd5-2cbe-42a1-be98-6b2e726b98de admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.105882 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cbcd5-2cbe-42a1-be98-6b2e726b98de admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.105980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cbcd5-2cbe-42a1-be98-6b2e726b98de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.107865 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cbcd5-2cbe-42a1-be98-6b2e726b98de admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.109686 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021835c-7068-4c29-a20c-2a30129e4a1d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.114797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85011b63-fd1f-4bec-9cd6-5c8c7cd3d9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.116185 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85011b63-fd1f-4bec-9cd6-5c8c7cd3d9ed admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.116589 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85011b63-fd1f-4bec-9cd6-5c8c7cd3d9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.116862 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85011b63-fd1f-4bec-9cd6-5c8c7cd3d9ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.117544 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85011b63-fd1f-4bec-9cd6-5c8c7cd3d9ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.119101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85011b63-fd1f-4bec-9cd6-5c8c7cd3d9ed admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.129201 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 791/1576] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:03.130377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27195c12-3825-4562-87ed-4d41c58f98a2 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.131276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27195c12-3825-4562-87ed-4d41c58f98a2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.131738 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27195c12-3825-4562-87ed-4d41c58f98a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.146821 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 786/1577] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:03 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:03.146821 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cbcd5-2cbe-42a1-be98-6b2e726b98de admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.159873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-27195c12-3825-4562-87ed-4d41c58f98a2 None None] MFA Rules not processed for user `6931920e71fc4cd69e0d654bf2b8badc`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:03.161115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-27195c12-3825-4562-87ed-4d41c58f98a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:03.209827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cbcd5-2cbe-42a1-be98-6b2e726b98de admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.214706 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ec1bd-e55e-4bb1-a5ea-3f47ae38efc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.215367 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ec1bd-e55e-4bb1-a5ea-3f47ae38efc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.215646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ec1bd-e55e-4bb1-a5ea-3f47ae38efc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.215922 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ec1bd-e55e-4bb1-a5ea-3f47ae38efc8 admin admin] PATH_INFO: `/v3/projects/7a6de534dbd541828ac211fae35b0ce4/users/1125a0b93c9d45ccb2d3f28f188dfd04/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.226176 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 792/1578] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:03.227134 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27195c12-3825-4562-87ed-4d41c58f98a2 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.232147 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93ec1bd-e55e-4bb1-a5ea-3f47ae38efc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a6de534dbd541828ac211fae35b0ce4, user_id=1125a0b93c9d45ccb2d3f28f188dfd04, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.234833 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93ec1bd-e55e-4bb1-a5ea-3f47ae38efc8 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.237498 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27195c12-3825-4562-87ed-4d41c58f98a2 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.243830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db830f49-8406-4c08-8f83-259b1af2f954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.244537 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db830f49-8406-4c08-8f83-259b1af2f954 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.244858 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db830f49-8406-4c08-8f83-259b1af2f954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.245165 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db830f49-8406-4c08-8f83-259b1af2f954 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.245848 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db830f49-8406-4c08-8f83-259b1af2f954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.247732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db830f49-8406-4c08-8f83-259b1af2f954 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.249509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 787/1579] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:03 2022] PUT /identity/v3/projects/7a6de534dbd541828ac211fae35b0ce4/users/1125a0b93c9d45ccb2d3f28f188dfd04/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:03.250465 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ec1bd-e55e-4bb1-a5ea-3f47ae38efc8 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.271015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 793/1580] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:03 2022] POST /identity/v3/users => generated 293 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:03.272246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db830f49-8406-4c08-8f83-259b1af2f954 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.281749 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db830f49-8406-4c08-8f83-259b1af2f954 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.312490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ec1bd-e55e-4bb1-a5ea-3f47ae38efc8 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.317015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993bea37-6fe6-47bf-879d-aeb936340fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.317628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993bea37-6fe6-47bf-879d-aeb936340fe8 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.317905 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993bea37-6fe6-47bf-879d-aeb936340fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.318206 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993bea37-6fe6-47bf-879d-aeb936340fe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.318863 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993bea37-6fe6-47bf-879d-aeb936340fe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.320793 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993bea37-6fe6-47bf-879d-aeb936340fe8 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.325946 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:03.340606 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d336e-62e2-42be-b1a7-39845de24375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.341232 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d336e-62e2-42be-b1a7-39845de24375 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.341512 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d336e-62e2-42be-b1a7-39845de24375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.341893 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d336e-62e2-42be-b1a7-39845de24375 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.342447 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d336e-62e2-42be-b1a7-39845de24375 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.344365 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d336e-62e2-42be-b1a7-39845de24375 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.351177 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 788/1581] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:03 2022] POST /identity/v3/projects => generated 393 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:03.351177 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993bea37-6fe6-47bf-879d-aeb936340fe8 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.373362 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 794/1582] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:03 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:03.373362 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d336e-62e2-42be-b1a7-39845de24375 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.406250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:03.412762 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993bea37-6fe6-47bf-879d-aeb936340fe8 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.439847 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d336e-62e2-42be-b1a7-39845de24375 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.445134 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1aeb72-73f2-4526-baa1-67b30d3436c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.445741 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1aeb72-73f2-4526-baa1-67b30d3436c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.446018 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1aeb72-73f2-4526-baa1-67b30d3436c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.446286 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1aeb72-73f2-4526-baa1-67b30d3436c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.446970 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1aeb72-73f2-4526-baa1-67b30d3436c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.448648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1aeb72-73f2-4526-baa1-67b30d3436c2 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.462550 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 795/1583] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:03.462550 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1aeb72-73f2-4526-baa1-67b30d3436c2 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.469891 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1aeb72-73f2-4526-baa1-67b30d3436c2 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.471614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5375c54-eeea-451c-b60d-eb6d3f1a4299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.472238 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5375c54-eeea-451c-b60d-eb6d3f1a4299 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.472546 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5375c54-eeea-451c-b60d-eb6d3f1a4299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.472818 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5375c54-eeea-451c-b60d-eb6d3f1a4299 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.473487 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5375c54-eeea-451c-b60d-eb6d3f1a4299 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.474476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d392adb-8959-47c1-87de-2af7b6a1ab3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.475054 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d392adb-8959-47c1-87de-2af7b6a1ab3f admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.475326 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d392adb-8959-47c1-87de-2af7b6a1ab3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.475521 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5375c54-eeea-451c-b60d-eb6d3f1a4299 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.475621 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d392adb-8959-47c1-87de-2af7b6a1ab3f admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.476247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d392adb-8959-47c1-87de-2af7b6a1ab3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.478182 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d392adb-8959-47c1-87de-2af7b6a1ab3f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.499394 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 796/1584] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:03 2022] POST /identity/v3/users => generated 290 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:03.500248 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d392adb-8959-47c1-87de-2af7b6a1ab3f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.504674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 789/1585] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:03 2022] POST /identity/v3/projects => generated 365 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:03.504674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7fcaa-9dcc-4b29-a151-1ee07f2d67a6 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.504674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7fcaa-9dcc-4b29-a151-1ee07f2d67a6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.504674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7fcaa-9dcc-4b29-a151-1ee07f2d67a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.510789 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d392adb-8959-47c1-87de-2af7b6a1ab3f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.522059 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-b9f7fcaa-9dcc-4b29-a151-1ee07f2d67a6 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:03.523243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b9f7fcaa-9dcc-4b29-a151-1ee07f2d67a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:03.567175 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791d647-a801-4409-8128-adfcdc42e3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.567713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791d647-a801-4409-8128-adfcdc42e3aa admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.567987 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791d647-a801-4409-8128-adfcdc42e3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.568350 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791d647-a801-4409-8128-adfcdc42e3aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.569035 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c791d647-a801-4409-8128-adfcdc42e3aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.570936 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c791d647-a801-4409-8128-adfcdc42e3aa admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.580532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 790/1586] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:03.581365 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7fcaa-9dcc-4b29-a151-1ee07f2d67a6 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.591036 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7fcaa-9dcc-4b29-a151-1ee07f2d67a6 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.596443 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99677830-6abc-4ec2-940a-e1eac0e52a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.597046 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99677830-6abc-4ec2-940a-e1eac0e52a4e admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.597325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99677830-6abc-4ec2-940a-e1eac0e52a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.597594 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99677830-6abc-4ec2-940a-e1eac0e52a4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.598273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99677830-6abc-4ec2-940a-e1eac0e52a4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.599944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99677830-6abc-4ec2-940a-e1eac0e52a4e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.603201 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 797/1587] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:03 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:03.605215 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791d647-a801-4409-8128-adfcdc42e3aa admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.612862 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 791/1588] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:03.612862 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99677830-6abc-4ec2-940a-e1eac0e52a4e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.621130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99677830-6abc-4ec2-940a-e1eac0e52a4e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.625775 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:03.667289 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791d647-a801-4409-8128-adfcdc42e3aa admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.672329 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615467b6-4814-464b-b95f-8eff49590682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.672980 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615467b6-4814-464b-b95f-8eff49590682 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.673260 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615467b6-4814-464b-b95f-8eff49590682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.673530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615467b6-4814-464b-b95f-8eff49590682 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.674213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615467b6-4814-464b-b95f-8eff49590682 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.676091 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615467b6-4814-464b-b95f-8eff49590682 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.677273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32d3c5-c7c7-4479-b687-4c42b5a69046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.678406 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32d3c5-c7c7-4479-b687-4c42b5a69046 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.678683 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32d3c5-c7c7-4479-b687-4c42b5a69046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.678946 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32d3c5-c7c7-4479-b687-4c42b5a69046 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.679592 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32d3c5-c7c7-4479-b687-4c42b5a69046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.681538 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b32d3c5-c7c7-4479-b687-4c42b5a69046 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.697543 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 798/1589] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:03 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:03.698412 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615467b6-4814-464b-b95f-8eff49590682 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.705462 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 792/1590] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:03 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:03.705462 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32d3c5-c7c7-4479-b687-4c42b5a69046 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.708483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615467b6-4814-464b-b95f-8eff49590682 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.714143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23324d-e1fb-490f-9439-53b936e40a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.714358 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32d3c5-c7c7-4479-b687-4c42b5a69046 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.714803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23324d-e1fb-490f-9439-53b936e40a5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.715067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23324d-e1fb-490f-9439-53b936e40a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.715342 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23324d-e1fb-490f-9439-53b936e40a5c admin admin] PATH_INFO: `/v3/projects/9ce267dd4f15447fae8aef5e99cbb018/users/257ddc018c33406aa47e7e7129943ae3/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.718911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874b303-d7d1-4c42-b0eb-8e4170f6245e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.719525 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874b303-d7d1-4c42-b0eb-8e4170f6245e admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.719807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874b303-d7d1-4c42-b0eb-8e4170f6245e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.720078 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874b303-d7d1-4c42-b0eb-8e4170f6245e admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.720796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874b303-d7d1-4c42-b0eb-8e4170f6245e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.722390 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874b303-d7d1-4c42-b0eb-8e4170f6245e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.729906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f23324d-e1fb-490f-9439-53b936e40a5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ce267dd4f15447fae8aef5e99cbb018, user_id=257ddc018c33406aa47e7e7129943ae3, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.732590 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f23324d-e1fb-490f-9439-53b936e40a5c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.732998 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 793/1591] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:03.733757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874b303-d7d1-4c42-b0eb-8e4170f6245e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.743257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:03.744787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874b303-d7d1-4c42-b0eb-8e4170f6245e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.750213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 799/1592] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:03 2022] PUT /identity/v3/projects/9ce267dd4f15447fae8aef5e99cbb018/users/257ddc018c33406aa47e7e7129943ae3/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:03.751101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23324d-e1fb-490f-9439-53b936e40a5c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.807019 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce87ed-b7d2-4bf4-82b4-121a25c367f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.807647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce87ed-b7d2-4bf4-82b4-121a25c367f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.807929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce87ed-b7d2-4bf4-82b4-121a25c367f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.808220 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce87ed-b7d2-4bf4-82b4-121a25c367f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.808900 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce87ed-b7d2-4bf4-82b4-121a25c367f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.810862 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce87ed-b7d2-4bf4-82b4-121a25c367f3 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.813743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23324d-e1fb-490f-9439-53b936e40a5c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.817410 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db13a8-e29f-4ce2-a5d1-1fd65a7a3499 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.817959 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db13a8-e29f-4ce2-a5d1-1fd65a7a3499 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.818280 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db13a8-e29f-4ce2-a5d1-1fd65a7a3499 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.818600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db13a8-e29f-4ce2-a5d1-1fd65a7a3499 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.828271 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db13a8-e29f-4ce2-a5d1-1fd65a7a3499 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.829741 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db13a8-e29f-4ce2-a5d1-1fd65a7a3499 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.831668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 794/1593] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:03 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:03.832496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce87ed-b7d2-4bf4-82b4-121a25c367f3 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.840259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 800/1594] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:03.840941 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db13a8-e29f-4ce2-a5d1-1fd65a7a3499 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.847788 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce87ed-b7d2-4bf4-82b4-121a25c367f3 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.850069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db13a8-e29f-4ce2-a5d1-1fd65a7a3499 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.852788 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d05e7f-b0db-47ad-b7a6-7dd4a2faf019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.853663 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d05e7f-b0db-47ad-b7a6-7dd4a2faf019 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.853942 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d05e7f-b0db-47ad-b7a6-7dd4a2faf019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.854216 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d05e7f-b0db-47ad-b7a6-7dd4a2faf019 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.854998 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d05e7f-b0db-47ad-b7a6-7dd4a2faf019 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.855124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0070e63-2a83-4b6e-92e8-7ffb3d84b183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.855711 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0070e63-2a83-4b6e-92e8-7ffb3d84b183 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.855990 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0070e63-2a83-4b6e-92e8-7ffb3d84b183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.856260 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0070e63-2a83-4b6e-92e8-7ffb3d84b183 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.857098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d05e7f-b0db-47ad-b7a6-7dd4a2faf019 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.857227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0070e63-2a83-4b6e-92e8-7ffb3d84b183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.858506 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0070e63-2a83-4b6e-92e8-7ffb3d84b183 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.873229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 801/1595] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:03.873229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0070e63-2a83-4b6e-92e8-7ffb3d84b183 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.881669 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0070e63-2a83-4b6e-92e8-7ffb3d84b183 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.885791 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 795/1596] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:03 2022] POST /identity/v3/users => generated 297 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:03.886565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab9e34-3b2d-4823-af7d-96007ae11aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.886803 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d05e7f-b0db-47ad-b7a6-7dd4a2faf019 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.887301 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab9e34-3b2d-4823-af7d-96007ae11aee admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.887642 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab9e34-3b2d-4823-af7d-96007ae11aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.887972 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab9e34-3b2d-4823-af7d-96007ae11aee admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.888766 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab9e34-3b2d-4823-af7d-96007ae11aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.890410 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab9e34-3b2d-4823-af7d-96007ae11aee admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.897801 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d05e7f-b0db-47ad-b7a6-7dd4a2faf019 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.904683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 802/1597] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:03.904683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab9e34-3b2d-4823-af7d-96007ae11aee admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.904935 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105cbec-5ad0-43e7-807f-9daaf2b292af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.904935 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105cbec-5ad0-43e7-807f-9daaf2b292af admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.904935 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105cbec-5ad0-43e7-807f-9daaf2b292af admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.904935 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105cbec-5ad0-43e7-807f-9daaf2b292af admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.905408 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9105cbec-5ad0-43e7-807f-9daaf2b292af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.907353 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9105cbec-5ad0-43e7-807f-9daaf2b292af admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.916076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab9e34-3b2d-4823-af7d-96007ae11aee admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.919570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 796/1598] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:03.921131 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c788d74-d0b4-4707-b6e7-ec43a4bfca59 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.921904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c788d74-d0b4-4707-b6e7-ec43a4bfca59 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.921904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c788d74-d0b4-4707-b6e7-ec43a4bfca59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.923970 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551e603-7730-4fcb-9b77-e5a5c344c942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.924619 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551e603-7730-4fcb-9b77-e5a5c344c942 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.925028 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551e603-7730-4fcb-9b77-e5a5c344c942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.925327 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551e603-7730-4fcb-9b77-e5a5c344c942 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.925988 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2551e603-7730-4fcb-9b77-e5a5c344c942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.927631 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2551e603-7730-4fcb-9b77-e5a5c344c942 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.939311 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 803/1599] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:03.940148 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551e603-7730-4fcb-9b77-e5a5c344c942 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.947962 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-6c788d74-d0b4-4707-b6e7-ec43a4bfca59 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:03.949873 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6c788d74-d0b4-4707-b6e7-ec43a4bfca59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:03.950628 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551e603-7730-4fcb-9b77-e5a5c344c942 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.958513 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 797/1600] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:03.961817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c788d74-d0b4-4707-b6e7-ec43a4bfca59 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.965529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119d18f-a8fb-406b-94d9-2e56b6b140b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.966437 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119d18f-a8fb-406b-94d9-2e56b6b140b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.966956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119d18f-a8fb-406b-94d9-2e56b6b140b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.967438 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119d18f-a8fb-406b-94d9-2e56b6b140b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.968386 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119d18f-a8fb-406b-94d9-2e56b6b140b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:03.970199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119d18f-a8fb-406b-94d9-2e56b6b140b2 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:03.973184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c788d74-d0b4-4707-b6e7-ec43a4bfca59 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:03.979072 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfcdee-6b8c-4583-b310-f5a277dfbea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:03.979862 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfcdee-6b8c-4583-b310-f5a277dfbea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:03.980172 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfcdee-6b8c-4583-b310-f5a277dfbea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:03.980445 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfcdee-6b8c-4583-b310-f5a277dfbea8 admin admin] PATH_INFO: `/v3/projects/923d72d5b3294f65aa6fdb0203731a27/users/b3c55244b9ed490eb71af80a976cd416/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:03.985131 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 804/1601] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:03.985131 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119d18f-a8fb-406b-94d9-2e56b6b140b2 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:03.997431 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119d18f-a8fb-406b-94d9-2e56b6b140b2 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:04.001838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bfcdee-6b8c-4583-b310-f5a277dfbea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=923d72d5b3294f65aa6fdb0203731a27, user_id=b3c55244b9ed490eb71af80a976cd416, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:04.003667 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd296fa0-d32a-4015-9353-c0a95f8011f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:04.004285 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd296fa0-d32a-4015-9353-c0a95f8011f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:04.004591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd296fa0-d32a-4015-9353-c0a95f8011f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:04.004863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd296fa0-d32a-4015-9353-c0a95f8011f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:04.004969 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bfcdee-6b8c-4583-b310-f5a277dfbea8 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:04.006086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd296fa0-d32a-4015-9353-c0a95f8011f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:04.008264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd296fa0-d32a-4015-9353-c0a95f8011f1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:04.021049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 805/1602] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:04.022358 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd296fa0-d32a-4015-9353-c0a95f8011f1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:04.043039 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 798/1603] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:03 2022] PUT /identity/v3/projects/923d72d5b3294f65aa6fdb0203731a27/users/b3c55244b9ed490eb71af80a976cd416/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:04.043039 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfcdee-6b8c-4583-b310-f5a277dfbea8 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:04.043612 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd296fa0-d32a-4015-9353-c0a95f8011f1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:04.070932 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:04.119660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a76f4e-20f0-40d9-a370-541823b82c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:04.120602 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a76f4e-20f0-40d9-a370-541823b82c93 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:04.120885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a76f4e-20f0-40d9-a370-541823b82c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:04.121169 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a76f4e-20f0-40d9-a370-541823b82c93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:04.121836 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a76f4e-20f0-40d9-a370-541823b82c93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:04.123474 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a76f4e-20f0-40d9-a370-541823b82c93 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:04.126925 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfcdee-6b8c-4583-b310-f5a277dfbea8 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:04.133290 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86820ef9-2d13-4e8b-897e-3bcf866b5be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:04.134137 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86820ef9-2d13-4e8b-897e-3bcf866b5be4 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:04.134588 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86820ef9-2d13-4e8b-897e-3bcf866b5be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:04.134728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 806/1604] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:04.135143 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86820ef9-2d13-4e8b-897e-3bcf866b5be4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:04.135593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a76f4e-20f0-40d9-a370-541823b82c93 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:04.136223 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86820ef9-2d13-4e8b-897e-3bcf866b5be4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:04.138135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86820ef9-2d13-4e8b-897e-3bcf866b5be4 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:04.146069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a76f4e-20f0-40d9-a370-541823b82c93 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:04.152527 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1f518-ff64-48a5-8bed-a7e151ad317d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:04.153193 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1f518-ff64-48a5-8bed-a7e151ad317d admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:04.153472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1f518-ff64-48a5-8bed-a7e151ad317d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:04.153742 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1f518-ff64-48a5-8bed-a7e151ad317d admin admin] PATH_INFO: `/v3/projects/6111202e81324d069fae641cc1606e77/users/79c2d686d2034765ac50f7ad75295197/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:04.154265 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 799/1605] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:04.155325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86820ef9-2d13-4e8b-897e-3bcf866b5be4 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:04.173176 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba1f518-ff64-48a5-8bed-a7e151ad317d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6111202e81324d069fae641cc1606e77, user_id=79c2d686d2034765ac50f7ad75295197, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:04.176072 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba1f518-ff64-48a5-8bed-a7e151ad317d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:04.200050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 807/1606] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:04 2022] PUT /identity/v3/projects/6111202e81324d069fae641cc1606e77/users/79c2d686d2034765ac50f7ad75295197/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:04.202369 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1f518-ff64-48a5-8bed-a7e151ad317d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:04.234159 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:04.234355 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86820ef9-2d13-4e8b-897e-3bcf866b5be4 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:04.283388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba1f518-ff64-48a5-8bed-a7e151ad317d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:04.289502 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd5bc2-e3e5-4ad8-880d-53bc8ed299ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:04.290368 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd5bc2-e3e5-4ad8-880d-53bc8ed299ef admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:04.290831 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd5bc2-e3e5-4ad8-880d-53bc8ed299ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:04.291329 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd5bc2-e3e5-4ad8-880d-53bc8ed299ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:04.292268 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd5bc2-e3e5-4ad8-880d-53bc8ed299ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:04.294188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd5bc2-e3e5-4ad8-880d-53bc8ed299ef admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:04.298834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bc9aa-a2ed-4ed1-b7cd-75f9450c8138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:04.299510 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bc9aa-a2ed-4ed1-b7cd-75f9450c8138 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:04.299793 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bc9aa-a2ed-4ed1-b7cd-75f9450c8138 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:04.300059 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bc9aa-a2ed-4ed1-b7cd-75f9450c8138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:04.307533 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 808/1607] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:04.308753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd5bc2-e3e5-4ad8-880d-53bc8ed299ef admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:04.315731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bc9aa-a2ed-4ed1-b7cd-75f9450c8138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:04.320124 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bc9aa-a2ed-4ed1-b7cd-75f9450c8138 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:04.324261 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd5bc2-e3e5-4ad8-880d-53bc8ed299ef admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:04.331019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8630f8-4f8d-4ba1-9afe-e10076a61df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:04.331611 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 800/1608] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:04.332792 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8630f8-4f8d-4ba1-9afe-e10076a61df0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:04.333315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8630f8-4f8d-4ba1-9afe-e10076a61df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:04.333562 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92e667-8d09-4e58-a48b-b538b76725ba None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:04.334523 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92e667-8d09-4e58-a48b-b538b76725ba None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:04.334872 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92e667-8d09-4e58-a48b-b538b76725ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:04.335139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8630f8-4f8d-4ba1-9afe-e10076a61df0 admin admin] PATH_INFO: `/v3/projects/f360267e7953431e87099a0c030e55a7/users/e7ac077f28514e85b9adcafb04e0dc0a/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:04.354150 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8630f8-4f8d-4ba1-9afe-e10076a61df0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f360267e7953431e87099a0c030e55a7, user_id=e7ac077f28514e85b9adcafb04e0dc0a, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:04.357171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8630f8-4f8d-4ba1-9afe-e10076a61df0 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:04.372646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-1b92e667-8d09-4e58-a48b-b538b76725ba None None] MFA Rules not processed for user `b3c55244b9ed490eb71af80a976cd416`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:04.373848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-1b92e667-8d09-4e58-a48b-b538b76725ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:04.377971 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 809/1609] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:04 2022] PUT /identity/v3/projects/f360267e7953431e87099a0c030e55a7/users/e7ac077f28514e85b9adcafb04e0dc0a/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:04.378931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8630f8-4f8d-4ba1-9afe-e10076a61df0 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:04.441489 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 801/1610] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:04.446870 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92e667-8d09-4e58-a48b-b538b76725ba None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:04.451519 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:04.467291 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8630f8-4f8d-4ba1-9afe-e10076a61df0 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:04.476513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8180c5-9dfa-4df5-97aa-ff7c7ec2b6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:04.477382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8180c5-9dfa-4df5-97aa-ff7c7ec2b6b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:04.477538 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8180c5-9dfa-4df5-97aa-ff7c7ec2b6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:04.477763 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8180c5-9dfa-4df5-97aa-ff7c7ec2b6b6 admin admin] PATH_INFO: `/v3/projects/6dea90217d154c2ab843c86c2a5ebb49/users/7c520a94e20245ca9cc898dee3ab89bc/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:04.498530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8180c5-9dfa-4df5-97aa-ff7c7ec2b6b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dea90217d154c2ab843c86c2a5ebb49, user_id=7c520a94e20245ca9cc898dee3ab89bc, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:04.501790 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8180c5-9dfa-4df5-97aa-ff7c7ec2b6b6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:04.522050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 810/1611] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:04 2022] PUT /identity/v3/projects/6dea90217d154c2ab843c86c2a5ebb49/users/7c520a94e20245ca9cc898dee3ab89bc/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:04.523498 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e7216-7e5e-4661-a266-dc73ca010dd4 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:04.525135 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e7216-7e5e-4661-a266-dc73ca010dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:04.525135 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e7216-7e5e-4661-a266-dc73ca010dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:04.538686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92e667-8d09-4e58-a48b-b538b76725ba None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:04.544830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b45e7216-7e5e-4661-a266-dc73ca010dd4 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:04.546207 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b45e7216-7e5e-4661-a266-dc73ca010dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:04.612949 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53451108-d87b-401f-be2e-ef6e8fc00f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:04.613577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53451108-d87b-401f-be2e-ef6e8fc00f55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:04.613826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53451108-d87b-401f-be2e-ef6e8fc00f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:04.614107 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53451108-d87b-401f-be2e-ef6e8fc00f55 admin admin] PATH_INFO: `/v3/projects/321f21add6a240a09f41cd78713a15be/users/8cbc932c57cc4dfd8abed78be68ff1c8/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:04.618036 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 811/1612] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:04.618921 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e7216-7e5e-4661-a266-dc73ca010dd4 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:04.631282 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e7216-7e5e-4661-a266-dc73ca010dd4 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:04.632278 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53451108-d87b-401f-be2e-ef6e8fc00f55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=321f21add6a240a09f41cd78713a15be, user_id=8cbc932c57cc4dfd8abed78be68ff1c8, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:04.635492 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53451108-d87b-401f-be2e-ef6e8fc00f55 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:04.636870 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d2c9d-8256-4d04-950b-72d8ba3611a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:04.637766 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d2c9d-8256-4d04-950b-72d8ba3611a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:04.637901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d2c9d-8256-4d04-950b-72d8ba3611a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:04.638076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d2c9d-8256-4d04-950b-72d8ba3611a7 admin admin] PATH_INFO: `/v3/projects/a5308b4ab1c94ae5b1735813b7d215aa/users/8c5ea1856efd4c0c9b4f615e1811b951/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:04.653294 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:04.657105 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 802/1613] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:04 2022] PUT /identity/v3/projects/321f21add6a240a09f41cd78713a15be/users/8cbc932c57cc4dfd8abed78be68ff1c8/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:04.657956 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53451108-d87b-401f-be2e-ef6e8fc00f55 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:04.661170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d2c9d-8256-4d04-950b-72d8ba3611a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5308b4ab1c94ae5b1735813b7d215aa, user_id=8c5ea1856efd4c0c9b4f615e1811b951, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:04.664035 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d2c9d-8256-4d04-950b-72d8ba3611a7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:04.696384 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 812/1614] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:04 2022] PUT /identity/v3/projects/a5308b4ab1c94ae5b1735813b7d215aa/users/8c5ea1856efd4c0c9b4f615e1811b951/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:04.696384 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6d0d9-6734-4826-b4ea-367c71236a3e None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:04.696384 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6d0d9-6734-4826-b4ea-367c71236a3e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:04.696384 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6d0d9-6734-4826-b4ea-367c71236a3e None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:04.696384 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 813/1615] 10.222.0.20 () {54 vars in 962 bytes} [Sat Aug 6 12:30:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:30:04.696384 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860efab8-92db-40ca-8dda-c2fc36df0b1b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:04.697107 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860efab8-92db-40ca-8dda-c2fc36df0b1b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:04.697107 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860efab8-92db-40ca-8dda-c2fc36df0b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:04.719819 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-860efab8-92db-40ca-8dda-c2fc36df0b1b None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:04.721185 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-860efab8-92db-40ca-8dda-c2fc36df0b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:04.771640 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53451108-d87b-401f-be2e-ef6e8fc00f55 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:04.797573 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 814/1616] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:04.798643 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860efab8-92db-40ca-8dda-c2fc36df0b1b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:04.820366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:04.845076 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ede785-5517-4b19-809e-63bded8aef61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:04.845771 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ede785-5517-4b19-809e-63bded8aef61 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:04.846283 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ede785-5517-4b19-809e-63bded8aef61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:04.846597 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ede785-5517-4b19-809e-63bded8aef61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:04.848198 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ede785-5517-4b19-809e-63bded8aef61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:04.851380 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ede785-5517-4b19-809e-63bded8aef61 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:04.868408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860efab8-92db-40ca-8dda-c2fc36df0b1b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:04.869087 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 803/1617] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:04.871327 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470af58c-194b-40ab-bcc4-3191181bdd09 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:04.871327 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470af58c-194b-40ab-bcc4-3191181bdd09 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:04.871530 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470af58c-194b-40ab-bcc4-3191181bdd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:04.872255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a84c4c-fcce-47d5-91bc-346df44e84bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:04.872978 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a84c4c-fcce-47d5-91bc-346df44e84bb service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:04.873350 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a84c4c-fcce-47d5-91bc-346df44e84bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:04.873686 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a84c4c-fcce-47d5-91bc-346df44e84bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:04.887670 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a84c4c-fcce-47d5-91bc-346df44e84bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:04.890161 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a84c4c-fcce-47d5-91bc-346df44e84bb service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:04.904196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 815/1618] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:04.905861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a6c80-5c8f-4b66-9aba-9bbbe65b78b9 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:04.906149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a6c80-5c8f-4b66-9aba-9bbbe65b78b9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:04.906435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a6c80-5c8f-4b66-9aba-9bbbe65b78b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:04.907272 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-470af58c-194b-40ab-bcc4-3191181bdd09 None None] MFA Rules not processed for user `8cbc932c57cc4dfd8abed78be68ff1c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:04.908688 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-470af58c-194b-40ab-bcc4-3191181bdd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:04.962932 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a62a6c80-5c8f-4b66-9aba-9bbbe65b78b9 None None] MFA Rules not processed for user `524b728bf63c4ca88e0f25e531cf5056`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:04.965503 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a62a6c80-5c8f-4b66-9aba-9bbbe65b78b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:05.046142 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 804/1619] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:05.046142 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727bfa0-a838-4d0c-8cae-4cb8cea435dc None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:05.046142 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727bfa0-a838-4d0c-8cae-4cb8cea435dc None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:05.046142 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727bfa0-a838-4d0c-8cae-4cb8cea435dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:05.084129 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 816/1620] 10.222.0.20 () {58 vars in 1059 bytes} [Sat Aug 6 12:30:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:05.085275 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a6c80-5c8f-4b66-9aba-9bbbe65b78b9 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:05.098110 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a6c80-5c8f-4b66-9aba-9bbbe65b78b9 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:05.105764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484f816-baef-4c78-8b91-493861c1e5be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:05.106643 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484f816-baef-4c78-8b91-493861c1e5be service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:05.106990 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484f816-baef-4c78-8b91-493861c1e5be service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:05.107435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484f816-baef-4c78-8b91-493861c1e5be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:05.111450 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-e727bfa0-a838-4d0c-8cae-4cb8cea435dc None None] MFA Rules not processed for user `8c5ea1856efd4c0c9b4f615e1811b951`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:05.114455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e727bfa0-a838-4d0c-8cae-4cb8cea435dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:05.120329 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484f816-baef-4c78-8b91-493861c1e5be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:05.122076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484f816-baef-4c78-8b91-493861c1e5be service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:05.126957 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:05.140789 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 817/1621] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:05.141720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484f816-baef-4c78-8b91-493861c1e5be service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:05.153220 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484f816-baef-4c78-8b91-493861c1e5be service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:05.159982 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a63b3b-5c96-4416-bf84-07e917ab1d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:05.160841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a63b3b-5c96-4416-bf84-07e917ab1d64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:05.161312 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a63b3b-5c96-4416-bf84-07e917ab1d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:05.161821 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a63b3b-5c96-4416-bf84-07e917ab1d64 admin admin] PATH_INFO: `/v3/projects/f360267e7953431e87099a0c030e55a7/users/e7ac077f28514e85b9adcafb04e0dc0a/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:05.164003 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a63b3b-5c96-4416-bf84-07e917ab1d64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f360267e7953431e87099a0c030e55a7, user_id=e7ac077f28514e85b9adcafb04e0dc0a, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:05.168540 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a63b3b-5c96-4416-bf84-07e917ab1d64 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:05.207134 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 818/1622] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:05 2022] PUT /identity/v3/projects/f360267e7953431e87099a0c030e55a7/users/e7ac077f28514e85b9adcafb04e0dc0a/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:05.211939 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e5b4b-f80a-46ee-9f12-25e24ca7e1e6 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:05.212250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e5b4b-f80a-46ee-9f12-25e24ca7e1e6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:05.214261 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e5b4b-f80a-46ee-9f12-25e24ca7e1e6 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:05.218480 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 819/1623] 10.222.0.20 () {54 vars in 962 bytes} [Sat Aug 6 12:30:05 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:30:05.221899 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e5b4b-f80a-46ee-9f12-25e24ca7e1e6 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:05.222173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 805/1624] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:05.223835 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd568d-3266-45bd-b599-41401170d2a2 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:05.224172 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd568d-3266-45bd-b599-41401170d2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:05.225110 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd568d-3266-45bd-b599-41401170d2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:05.263477 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-9edd568d-3266-45bd-b599-41401170d2a2 None None] MFA Rules not processed for user `e7ac077f28514e85b9adcafb04e0dc0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:05.264826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9edd568d-3266-45bd-b599-41401170d2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:05.328140 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:05.385283 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 806/1625] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:05.387318 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8274eff-60f2-45dc-bf7f-0b817b941008 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:05.387610 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8274eff-60f2-45dc-bf7f-0b817b941008 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:05.387951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8274eff-60f2-45dc-bf7f-0b817b941008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:05.417381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c8274eff-60f2-45dc-bf7f-0b817b941008 None None] MFA Rules not processed for user `524b728bf63c4ca88e0f25e531cf5056`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:05.418727 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c8274eff-60f2-45dc-bf7f-0b817b941008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:05.434643 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e5b4b-f80a-46ee-9f12-25e24ca7e1e6 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:05.436380 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 807/1626] 10.222.0.20 () {58 vars in 1059 bytes} [Sat Aug 6 12:30:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:05.437290 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8274eff-60f2-45dc-bf7f-0b817b941008 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:05.459493 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2d55a-a20b-484b-9d37-63a8266c6878 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:05.460118 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2d55a-a20b-484b-9d37-63a8266c6878 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:05.462150 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2d55a-a20b-484b-9d37-63a8266c6878 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:05.462150 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2d55a-a20b-484b-9d37-63a8266c6878 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:05.579533 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8274eff-60f2-45dc-bf7f-0b817b941008 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:05.582631 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2d55a-a20b-484b-9d37-63a8266c6878 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:05.584526 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8716e8-b6ee-4a81-8b60-65ab5d69e3e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:05.585173 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2d55a-a20b-484b-9d37-63a8266c6878 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:05.586693 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8716e8-b6ee-4a81-8b60-65ab5d69e3e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:05.587011 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8716e8-b6ee-4a81-8b60-65ab5d69e3e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:05.587485 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8716e8-b6ee-4a81-8b60-65ab5d69e3e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:05.602222 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 820/1627] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:05.603763 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2d55a-a20b-484b-9d37-63a8266c6878 service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:05.618137 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2d55a-a20b-484b-9d37-63a8266c6878 service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:05.624194 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c0160-fa00-4120-ad5e-f35d8a09ef19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:05.625218 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c0160-fa00-4120-ad5e-f35d8a09ef19 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:05.625970 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c0160-fa00-4120-ad5e-f35d8a09ef19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:05.627807 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c0160-fa00-4120-ad5e-f35d8a09ef19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:05.641866 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c0160-fa00-4120-ad5e-f35d8a09ef19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:05.644321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c0160-fa00-4120-ad5e-f35d8a09ef19 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:05.657852 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 821/1628] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:05.658338 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8716e8-b6ee-4a81-8b60-65ab5d69e3e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:05.660017 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8716e8-b6ee-4a81-8b60-65ab5d69e3e5 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:05.673368 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c0160-fa00-4120-ad5e-f35d8a09ef19 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:05.674395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 808/1629] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:05.713805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c0160-fa00-4120-ad5e-f35d8a09ef19 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:05.726155 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944a7ba-dd83-45ad-8091-5e54b7ae4d47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:05.726810 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944a7ba-dd83-45ad-8091-5e54b7ae4d47 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:05.727090 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944a7ba-dd83-45ad-8091-5e54b7ae4d47 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:05.727470 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944a7ba-dd83-45ad-8091-5e54b7ae4d47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:05.848400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9944a7ba-dd83-45ad-8091-5e54b7ae4d47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:05.851524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9944a7ba-dd83-45ad-8091-5e54b7ae4d47 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:05.868827 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 822/1630] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:05.908288 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8716e8-b6ee-4a81-8b60-65ab5d69e3e5 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:05.918440 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8716e8-b6ee-4a81-8b60-65ab5d69e3e5 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:05.920353 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafcad6-a063-4758-b944-d12fa48553ed None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:05.920775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafcad6-a063-4758-b944-d12fa48553ed None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:05.921224 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafcad6-a063-4758-b944-d12fa48553ed None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:05.922976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 823/1631] 10.222.0.20 () {54 vars in 912 bytes} [Sat Aug 6 12:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 06 12:30:05.930411 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd2233-4d45-476d-94af-552155a5d66c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:05.930832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd2233-4d45-476d-94af-552155a5d66c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:05.931027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1e307-8d0a-4719-b601-bc4bfddf890a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:05.931306 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd2233-4d45-476d-94af-552155a5d66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:05.931660 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1e307-8d0a-4719-b601-bc4bfddf890a service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:05.931938 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1e307-8d0a-4719-b601-bc4bfddf890a service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:05.932212 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1e307-8d0a-4719-b601-bc4bfddf890a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:05.949353 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:05.961149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-fdfd2233-4d45-476d-94af-552155a5d66c None None] MFA Rules not processed for user `524b728bf63c4ca88e0f25e531cf5056`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:05.962647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fdfd2233-4d45-476d-94af-552155a5d66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:05.981526 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 824/1632] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Aug 6 12:30:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:06.079687 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc1e307-8d0a-4719-b601-bc4bfddf890a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:06.081196 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc1e307-8d0a-4719-b601-bc4bfddf890a service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:06.106104 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 809/1633] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:06.433345 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccd260-fbaf-4ce1-a474-09cca0a4c99a None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:06.433345 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccd260-fbaf-4ce1-a474-09cca0a4c99a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:06.433345 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccd260-fbaf-4ce1-a474-09cca0a4c99a None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:06.433345 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 825/1634] 10.222.0.20 () {54 vars in 912 bytes} [Sat Aug 6 12:30:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:30:06.447401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c13d1-4ba7-4ca2-ae73-630798ffae12 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:06.447401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c13d1-4ba7-4ca2-ae73-630798ffae12 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:06.447401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c13d1-4ba7-4ca2-ae73-630798ffae12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:06.447401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-fa5c13d1-4ba7-4ca2-ae73-630798ffae12 None None] MFA Rules not processed for user `524b728bf63c4ca88e0f25e531cf5056`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:06.447401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fa5c13d1-4ba7-4ca2-ae73-630798ffae12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:06.447401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 810/1635] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Aug 6 12:30:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:06.495813 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ccd260-fbaf-4ce1-a474-09cca0a4c99a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:06.519298 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ccd260-fbaf-4ce1-a474-09cca0a4c99a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:06.523895 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de9c92-c169-4e6e-ae34-88a9d50e5f9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:06.525057 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de9c92-c169-4e6e-ae34-88a9d50e5f9c service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:06.525057 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de9c92-c169-4e6e-ae34-88a9d50e5f9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:06.525057 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de9c92-c169-4e6e-ae34-88a9d50e5f9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:06.547818 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de9c92-c169-4e6e-ae34-88a9d50e5f9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:06.549395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de9c92-c169-4e6e-ae34-88a9d50e5f9c service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:06.575608 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 826/1636] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:06.710756 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c13d1-4ba7-4ca2-ae73-630798ffae12 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:06.741680 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c13d1-4ba7-4ca2-ae73-630798ffae12 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:06.757227 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4390f1-a99f-4d19-89f5-4a9308c506c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:06.760838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4390f1-a99f-4d19-89f5-4a9308c506c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:06.761085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4390f1-a99f-4d19-89f5-4a9308c506c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:06.761370 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4390f1-a99f-4d19-89f5-4a9308c506c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:06.805496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4390f1-a99f-4d19-89f5-4a9308c506c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:06.813383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4390f1-a99f-4d19-89f5-4a9308c506c0 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:06.870228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 811/1637] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:16.109233 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a2115-80af-4fd1-b408-9e3ede8d9c0d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:16.109952 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a2115-80af-4fd1-b408-9e3ede8d9c0d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:16.109952 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a2115-80af-4fd1-b408-9e3ede8d9c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:16.187553 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-757a2115-80af-4fd1-b408-9e3ede8d9c0d None None] MFA Rules not processed for user `257ddc018c33406aa47e7e7129943ae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:16.188821 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-757a2115-80af-4fd1-b408-9e3ede8d9c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:16.366323 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 827/1638] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:16 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:16.374402 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4390f1-a99f-4d19-89f5-4a9308c506c0 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:16.403603 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4390f1-a99f-4d19-89f5-4a9308c506c0 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:16.432258 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c813ea-eab6-43c8-9909-b6e84d7a2953 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:16.437736 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c813ea-eab6-43c8-9909-b6e84d7a2953 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:16.439290 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c813ea-eab6-43c8-9909-b6e84d7a2953 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:16.449459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c813ea-eab6-43c8-9909-b6e84d7a2953 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:16.497453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:16.509779 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c813ea-eab6-43c8-9909-b6e84d7a2953 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:16.512967 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c813ea-eab6-43c8-9909-b6e84d7a2953 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:16.551581 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 812/1639] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:16.988878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95e97c-1400-446b-bfb3-75732ae45050 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:16.989162 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95e97c-1400-446b-bfb3-75732ae45050 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:16.989493 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95e97c-1400-446b-bfb3-75732ae45050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:17.060068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4c95e97c-1400-446b-bfb3-75732ae45050 None None] MFA Rules not processed for user `1125a0b93c9d45ccb2d3f28f188dfd04`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:17.061432 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4c95e97c-1400-446b-bfb3-75732ae45050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:17.217947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 828/1640] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:17.234501 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c813ea-eab6-43c8-9909-b6e84d7a2953 service nova] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:17.256034 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c813ea-eab6-43c8-9909-b6e84d7a2953 service nova] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:17.278084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c9963-28da-4dbe-9c9c-01b8f97bf302 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:17.279461 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c9963-28da-4dbe-9c9c-01b8f97bf302 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:17.279461 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c9963-28da-4dbe-9c9c-01b8f97bf302 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:17.279889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c9963-28da-4dbe-9c9c-01b8f97bf302 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:17.298591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349c9963-28da-4dbe-9c9c-01b8f97bf302 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:17.301618 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349c9963-28da-4dbe-9c9c-01b8f97bf302 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:17.329893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 813/1641] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:17 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:17.893713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11795f-986e-4a10-a18b-cd222a2001c3 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:17.893713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11795f-986e-4a10-a18b-cd222a2001c3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:17.894321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11795f-986e-4a10-a18b-cd222a2001c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:17.966568 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4a11795f-986e-4a10-a18b-cd222a2001c3 None None] MFA Rules not processed for user `79c2d686d2034765ac50f7ad75295197`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:17.970281 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4a11795f-986e-4a10-a18b-cd222a2001c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:18.099226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:18.118960 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 829/1642] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:18.129497 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e44c8-46be-4a5a-96ab-7da2facc66d3 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:18.130044 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e44c8-46be-4a5a-96ab-7da2facc66d3 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:18.130259 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e44c8-46be-4a5a-96ab-7da2facc66d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:18.171979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-a65e44c8-46be-4a5a-96ab-7da2facc66d3 None None] MFA Rules not processed for user `79c2d686d2034765ac50f7ad75295197`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:18.173456 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a65e44c8-46be-4a5a-96ab-7da2facc66d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:18.193747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 814/1643] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:18.206293 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11795f-986e-4a10-a18b-cd222a2001c3 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:18.231723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11795f-986e-4a10-a18b-cd222a2001c3 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:18.249254 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f46023-3127-4627-ba77-a14b515edbce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:18.249966 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f46023-3127-4627-ba77-a14b515edbce service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:18.250332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f46023-3127-4627-ba77-a14b515edbce service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:18.252080 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f46023-3127-4627-ba77-a14b515edbce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:18.278902 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f46023-3127-4627-ba77-a14b515edbce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:18.280880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f46023-3127-4627-ba77-a14b515edbce service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:18.309252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 830/1644] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:19.521076 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e65346-9f44-40fc-a49b-37bc072cf187 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:19.521978 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e65346-9f44-40fc-a49b-37bc072cf187 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:19.522898 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e65346-9f44-40fc-a49b-37bc072cf187 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:19.531901 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 815/1645] 10.222.0.20 () {54 vars in 912 bytes} [Sat Aug 6 12:30:19 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:30:19.538901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be25b6-2ed7-4abc-a2a5-7cb758a177e3 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:19.539345 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be25b6-2ed7-4abc-a2a5-7cb758a177e3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:19.539443 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be25b6-2ed7-4abc-a2a5-7cb758a177e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:19.597356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3ff58-0442-4044-8548-d7198842b8f3 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:19.598054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3ff58-0442-4044-8548-d7198842b8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:19.598565 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3ff58-0442-4044-8548-d7198842b8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:19.622834 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-36be25b6-2ed7-4abc-a2a5-7cb758a177e3 None None] MFA Rules not processed for user `55563aaa662246c5aaf107145475bd76`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:19.625636 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-36be25b6-2ed7-4abc-a2a5-7cb758a177e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:19.722170 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-94e3ff58-0442-4044-8548-d7198842b8f3 None None] MFA Rules not processed for user `7c520a94e20245ca9cc898dee3ab89bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:19.731138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-94e3ff58-0442-4044-8548-d7198842b8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:19.782848 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 831/1646] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Aug 6 12:30:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:19.819480 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be25b6-2ed7-4abc-a2a5-7cb758a177e3 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:19.832522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be25b6-2ed7-4abc-a2a5-7cb758a177e3 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:19.849612 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228da6c8-f416-4c5e-b34a-0ebb8ba4ca5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:19.850729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228da6c8-f416-4c5e-b34a-0ebb8ba4ca5c service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:19.850845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228da6c8-f416-4c5e-b34a-0ebb8ba4ca5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:19.851209 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228da6c8-f416-4c5e-b34a-0ebb8ba4ca5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:19.893147 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228da6c8-f416-4c5e-b34a-0ebb8ba4ca5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:19.895930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228da6c8-f416-4c5e-b34a-0ebb8ba4ca5c service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:19.948028 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 832/1647] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:20.041940 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c0f5e-6bc5-4549-aa40-61585ee2681c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:20.042268 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c0f5e-6bc5-4549-aa40-61585ee2681c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:20.042719 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 816/1648] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:19 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 444 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:20.042848 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c0f5e-6bc5-4549-aa40-61585ee2681c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:20.093156 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e87c0f5e-6bc5-4549-aa40-61585ee2681c None None] MFA Rules not processed for user `7c520a94e20245ca9cc898dee3ab89bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:20.099084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e87c0f5e-6bc5-4549-aa40-61585ee2681c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:20.107938 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 833/1649] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:20.128923 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3ff58-0442-4044-8548-d7198842b8f3 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:20.154216 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3ff58-0442-4044-8548-d7198842b8f3 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:20.171945 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a8872-d9c5-40db-86ab-56c05922ac48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:20.173491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a8872-d9c5-40db-86ab-56c05922ac48 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:20.173491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a8872-d9c5-40db-86ab-56c05922ac48 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:20.176383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a8872-d9c5-40db-86ab-56c05922ac48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:20.195925 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a8872-d9c5-40db-86ab-56c05922ac48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:20.197697 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a8872-d9c5-40db-86ab-56c05922ac48 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:20.212508 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 817/1650] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:20 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:20.819670 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62a557-1c26-4da4-a14c-b6073e95fd6d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:20.820735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62a557-1c26-4da4-a14c-b6073e95fd6d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:20.821199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62a557-1c26-4da4-a14c-b6073e95fd6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:20.871504 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-aa62a557-1c26-4da4-a14c-b6073e95fd6d None None] MFA Rules not processed for user `257ddc018c33406aa47e7e7129943ae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:20.875100 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-aa62a557-1c26-4da4-a14c-b6073e95fd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:20.899033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 834/1651] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:20 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:21.236049 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54a540-61ea-4fa8-8e8e-573844ab3886 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:21.236482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54a540-61ea-4fa8-8e8e-573844ab3886 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:21.236482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54a540-61ea-4fa8-8e8e-573844ab3886 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:21.238431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 818/1652] 10.222.0.20 () {54 vars in 961 bytes} [Sat Aug 6 12:30:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:30:21.245670 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20ecb1-d60a-436f-b041-c98bf4423015 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:21.246236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20ecb1-d60a-436f-b041-c98bf4423015 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:21.246716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20ecb1-d60a-436f-b041-c98bf4423015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:21.291277 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ff20ecb1-d60a-436f-b041-c98bf4423015 None None] MFA Rules not processed for user `e2244655ba304b9c980b5e68772eefc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:21.293044 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ff20ecb1-d60a-436f-b041-c98bf4423015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:21.437383 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 835/1653] 10.222.0.20 () {58 vars in 1058 bytes} [Sat Aug 6 12:30:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:30:21.454166 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54a540-61ea-4fa8-8e8e-573844ab3886 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:21.503881 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54a540-61ea-4fa8-8e8e-573844ab3886 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:21.519029 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97748fcb-f4d3-4f90-94f4-f47f284727d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2244655ba304b9c980b5e68772eefc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:21.519696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97748fcb-f4d3-4f90-94f4-f47f284727d6 service placement] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:21.519987 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97748fcb-f4d3-4f90-94f4-f47f284727d6 service placement] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:21.520289 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97748fcb-f4d3-4f90-94f4-f47f284727d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:21.533966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-97748fcb-f4d3-4f90-94f4-f47f284727d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:21.557611 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97748fcb-f4d3-4f90-94f4-f47f284727d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:21.559289 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97748fcb-f4d3-4f90-94f4-f47f284727d6 service placement] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:21.583808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 819/1654] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:21.816893 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490c1d7-dcce-498f-bf88-1f41e057ef2d None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:21.817174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490c1d7-dcce-498f-bf88-1f41e057ef2d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:21.817684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490c1d7-dcce-498f-bf88-1f41e057ef2d None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:21.819186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 836/1655] 10.222.0.20 () {54 vars in 912 bytes} [Sat Aug 6 12:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:30:21.829790 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d4d7d-bfe5-40fc-a33a-e4ac1d4d2914 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:21.830161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d4d7d-bfe5-40fc-a33a-e4ac1d4d2914 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:21.830455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d4d7d-bfe5-40fc-a33a-e4ac1d4d2914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:21.897520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-e26d4d7d-bfe5-40fc-a33a-e4ac1d4d2914 None None] MFA Rules not processed for user `55563aaa662246c5aaf107145475bd76`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:21.898826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e26d4d7d-bfe5-40fc-a33a-e4ac1d4d2914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:21.929953 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 820/1656] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Aug 6 12:30:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:21.983383 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490c1d7-dcce-498f-bf88-1f41e057ef2d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:22.005598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490c1d7-dcce-498f-bf88-1f41e057ef2d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:22.026124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91e9db-a4c9-43ae-8ec9-746c9baeb9fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:22.026775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91e9db-a4c9-43ae-8ec9-746c9baeb9fa service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:22.027147 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91e9db-a4c9-43ae-8ec9-746c9baeb9fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:22.027632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91e9db-a4c9-43ae-8ec9-746c9baeb9fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:22.071196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91e9db-a4c9-43ae-8ec9-746c9baeb9fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:22.080595 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91e9db-a4c9-43ae-8ec9-746c9baeb9fa service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:22.106684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 837/1657] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:23.158151 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337e81a-4ce1-4e90-8a53-210e2bce5081 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:23.158151 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337e81a-4ce1-4e90-8a53-210e2bce5081 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:23.158590 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337e81a-4ce1-4e90-8a53-210e2bce5081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:23.256134 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91e9db-a4c9-43ae-8ec9-746c9baeb9fa service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:23.266246 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-6337e81a-4ce1-4e90-8a53-210e2bce5081 None None] MFA Rules not processed for user `1125a0b93c9d45ccb2d3f28f188dfd04`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:23.277300 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6337e81a-4ce1-4e90-8a53-210e2bce5081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:23.302809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 821/1658] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:23.385856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:23.525955 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91e9db-a4c9-43ae-8ec9-746c9baeb9fa service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:23.537777 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5243604c-401a-4d66-b12e-8abfe8c99c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:23.542095 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243604c-401a-4d66-b12e-8abfe8c99c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:23.543044 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243604c-401a-4d66-b12e-8abfe8c99c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:23.543550 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243604c-401a-4d66-b12e-8abfe8c99c6b admin admin] PATH_INFO: `/v3/users/8cbc932c57cc4dfd8abed78be68ff1c8` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:23.546768 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5243604c-401a-4d66-b12e-8abfe8c99c6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cbc932c57cc4dfd8abed78be68ff1c8)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:23.566183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5243604c-401a-4d66-b12e-8abfe8c99c6b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:23.788731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5243604c-401a-4d66-b12e-8abfe8c99c6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cbc932c57cc4dfd8abed78be68ff1c8'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:23.841427 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5243604c-401a-4d66-b12e-8abfe8c99c6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cbc932c57cc4dfd8abed78be68ff1c8'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:23.844433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5243604c-401a-4d66-b12e-8abfe8c99c6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cbc932c57cc4dfd8abed78be68ff1c8'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:23.845566 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5243604c-401a-4d66-b12e-8abfe8c99c6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cbc932c57cc4dfd8abed78be68ff1c8'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:23.872827 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5243604c-401a-4d66-b12e-8abfe8c99c6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cbc932c57cc4dfd8abed78be68ff1c8'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:23.932632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 838/1659] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:30:23 2022] DELETE /identity/v3/users/8cbc932c57cc4dfd8abed78be68ff1c8 => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:23.932810 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d5e0c-e92c-4d1c-8b3a-71bf5f3d185a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:23.932810 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d5e0c-e92c-4d1c-8b3a-71bf5f3d185a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:23.932810 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d5e0c-e92c-4d1c-8b3a-71bf5f3d185a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:23.975974 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-345d5e0c-e92c-4d1c-8b3a-71bf5f3d185a None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:23.977535 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-345d5e0c-e92c-4d1c-8b3a-71bf5f3d185a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:24.000695 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee1d90-fa46-40c5-9024-d6c266d06c8d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:24.000695 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee1d90-fa46-40c5-9024-d6c266d06c8d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:24.000695 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee1d90-fa46-40c5-9024-d6c266d06c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:24.026354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c7ee1d90-fa46-40c5-9024-d6c266d06c8d None None] MFA Rules not processed for user `7c520a94e20245ca9cc898dee3ab89bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:24.028044 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c7ee1d90-fa46-40c5-9024-d6c266d06c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:24.211087 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 839/1660] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:30:24.227897 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 822/1661] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:24.241285 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee1d90-fa46-40c5-9024-d6c266d06c8d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:24.244490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c7ee1d90-fa46-40c5-9024-d6c266d06c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:24.275824 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d5e0c-e92c-4d1c-8b3a-71bf5f3d185a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:24.277424 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-345d5e0c-e92c-4d1c-8b3a-71bf5f3d185a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:24.335651 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d5e0c-e92c-4d1c-8b3a-71bf5f3d185a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:24.356784 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f347dc8-d400-47fe-85b5-cc5b85cabdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:24.357727 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f347dc8-d400-47fe-85b5-cc5b85cabdb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:24.358146 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f347dc8-d400-47fe-85b5-cc5b85cabdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:24.358554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f347dc8-d400-47fe-85b5-cc5b85cabdb6 admin admin] PATH_INFO: `/v3/users/6931920e71fc4cd69e0d654bf2b8badc` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:24.361099 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f347dc8-d400-47fe-85b5-cc5b85cabdb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6931920e71fc4cd69e0d654bf2b8badc)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:24.362153 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: WARNING py.warnings [None req-3f347dc8-d400-47fe-85b5-cc5b85cabdb6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 12:30:24.362153 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: warnings.warn(msg) Aug 06 12:30:24.362153 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: Aug 06 12:30:24.363370 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f347dc8-d400-47fe-85b5-cc5b85cabdb6 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:24.389027 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee1d90-fa46-40c5-9024-d6c266d06c8d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:24.420202 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5711f-c96b-4969-9a4a-80f18dd8c928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:24.420878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5711f-c96b-4969-9a4a-80f18dd8c928 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:24.421317 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5711f-c96b-4969-9a4a-80f18dd8c928 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:24.421759 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5711f-c96b-4969-9a4a-80f18dd8c928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:24.448764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd5711f-c96b-4969-9a4a-80f18dd8c928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:24.450472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd5711f-c96b-4969-9a4a-80f18dd8c928 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:24.496391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 840/1662] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:24.500525 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:24.613710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3f347dc8-d400-47fe-85b5-cc5b85cabdb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6931920e71fc4cd69e0d654bf2b8badc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:24.648634 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3f347dc8-d400-47fe-85b5-cc5b85cabdb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6931920e71fc4cd69e0d654bf2b8badc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:24.731606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3f347dc8-d400-47fe-85b5-cc5b85cabdb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6931920e71fc4cd69e0d654bf2b8badc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:24.732436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3f347dc8-d400-47fe-85b5-cc5b85cabdb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6931920e71fc4cd69e0d654bf2b8badc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:24.733832 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3f347dc8-d400-47fe-85b5-cc5b85cabdb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6931920e71fc4cd69e0d654bf2b8badc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:24.810583 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 823/1663] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:30:24 2022] DELETE /identity/v3/users/6931920e71fc4cd69e0d654bf2b8badc => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:30:24.815434 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634e21e-814d-4720-8e28-1295b9a93adc None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:24.815715 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634e21e-814d-4720-8e28-1295b9a93adc None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:24.816258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634e21e-814d-4720-8e28-1295b9a93adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:24.826155 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:24.880015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4634e21e-814d-4720-8e28-1295b9a93adc None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:24.889644 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4634e21e-814d-4720-8e28-1295b9a93adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:25.146540 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 841/1664] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:25.246410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f347dc8-d400-47fe-85b5-cc5b85cabdb6 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:25.248032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3f347dc8-d400-47fe-85b5-cc5b85cabdb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:25.452435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d79a13-4b03-408a-9b2d-7a40c5f5696a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:25.453034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d79a13-4b03-408a-9b2d-7a40c5f5696a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:25.453197 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d79a13-4b03-408a-9b2d-7a40c5f5696a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:25.491664 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f347dc8-d400-47fe-85b5-cc5b85cabdb6 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:25.512165 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06080811-5d2b-495b-a182-28a3e6100036 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:25.517339 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06080811-5d2b-495b-a182-28a3e6100036 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:25.521022 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06080811-5d2b-495b-a182-28a3e6100036 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:25.521760 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06080811-5d2b-495b-a182-28a3e6100036 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:25.566382 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06080811-5d2b-495b-a182-28a3e6100036 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:25.572531 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06080811-5d2b-495b-a182-28a3e6100036 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:25.612147 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 824/1665] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:25.651624 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d0d79a13-4b03-408a-9b2d-7a40c5f5696a None None] MFA Rules not processed for user `c810b52785b94c0cab4fd9f0d6e65825`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:25.655266 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d0d79a13-4b03-408a-9b2d-7a40c5f5696a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:25.793373 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 842/1666] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Aug 6 12:30:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:30:26.028020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06080811-5d2b-495b-a182-28a3e6100036 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:26.029583 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-06080811-5d2b-495b-a182-28a3e6100036 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:26.272921 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06080811-5d2b-495b-a182-28a3e6100036 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:26.289665 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789277b4-ecab-4c32-80e4-99b04f82a419 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efc792a2655439f8b8070010059b6bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:26.296264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789277b4-ecab-4c32-80e4-99b04f82a419 service swift] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:26.299324 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789277b4-ecab-4c32-80e4-99b04f82a419 service swift] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:26.301444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789277b4-ecab-4c32-80e4-99b04f82a419 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:26.323534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789277b4-ecab-4c32-80e4-99b04f82a419 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:26.330647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789277b4-ecab-4c32-80e4-99b04f82a419 service swift] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:26.339277 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d79a13-4b03-408a-9b2d-7a40c5f5696a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:26.340886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d0d79a13-4b03-408a-9b2d-7a40c5f5696a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:26.360529 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 825/1667] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Aug 6 12:30:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 12:30:26.398171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d79a13-4b03-408a-9b2d-7a40c5f5696a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:26.421840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11358715-f407-4e74-919f-57d49d575c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:26.422613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11358715-f407-4e74-919f-57d49d575c0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:26.422978 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11358715-f407-4e74-919f-57d49d575c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:26.424391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11358715-f407-4e74-919f-57d49d575c0d admin admin] PATH_INFO: `/v3/users/b3c55244b9ed490eb71af80a976cd416` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:26.433944 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11358715-f407-4e74-919f-57d49d575c0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3c55244b9ed490eb71af80a976cd416)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:26.436082 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11358715-f407-4e74-919f-57d49d575c0d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:26.703796 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-11358715-f407-4e74-919f-57d49d575c0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3c55244b9ed490eb71af80a976cd416'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:26.737948 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-11358715-f407-4e74-919f-57d49d575c0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3c55244b9ed490eb71af80a976cd416'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:26.739802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-11358715-f407-4e74-919f-57d49d575c0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3c55244b9ed490eb71af80a976cd416'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:26.741780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-11358715-f407-4e74-919f-57d49d575c0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3c55244b9ed490eb71af80a976cd416'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:26.751693 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:26.760219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-11358715-f407-4e74-919f-57d49d575c0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3c55244b9ed490eb71af80a976cd416'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:26.817425 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 843/1668] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:30:26 2022] DELETE /identity/v3/users/b3c55244b9ed490eb71af80a976cd416 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:26.822171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7d5ec-86f0-410f-83ff-ceccf0c8dd22 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:26.822457 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7d5ec-86f0-410f-83ff-ceccf0c8dd22 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:26.822783 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7d5ec-86f0-410f-83ff-ceccf0c8dd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:26.914766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3ba7d5ec-86f0-410f-83ff-ceccf0c8dd22 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:26.924866 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3ba7d5ec-86f0-410f-83ff-ceccf0c8dd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:26.943689 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:27.133210 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 826/1669] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:27.177624 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11358715-f407-4e74-919f-57d49d575c0d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:27.183847 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-11358715-f407-4e74-919f-57d49d575c0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:27.278182 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7d5ec-86f0-410f-83ff-ceccf0c8dd22 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:27.280104 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3ba7d5ec-86f0-410f-83ff-ceccf0c8dd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:27.323347 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7d5ec-86f0-410f-83ff-ceccf0c8dd22 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:27.346113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9645b4f4-8cb8-4f50-8dff-24da2f4dadca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:27.346704 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9645b4f4-8cb8-4f50-8dff-24da2f4dadca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:27.346983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9645b4f4-8cb8-4f50-8dff-24da2f4dadca admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:27.347255 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9645b4f4-8cb8-4f50-8dff-24da2f4dadca admin admin] PATH_INFO: `/v3/projects/321f21add6a240a09f41cd78713a15be` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:27.355681 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9645b4f4-8cb8-4f50-8dff-24da2f4dadca admin admin] RBAC: Authorizing `identity:delete_project(project_id=321f21add6a240a09f41cd78713a15be)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:27.388571 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9645b4f4-8cb8-4f50-8dff-24da2f4dadca admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:27.527818 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11358715-f407-4e74-919f-57d49d575c0d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:27.613291 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9645b4f4-8cb8-4f50-8dff-24da2f4dadca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '321f21add6a240a09f41cd78713a15be', 'request_id': 'req-9645b4f4-8cb8-4f50-8dff-24da2f4dadca'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:27.630839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9645b4f4-8cb8-4f50-8dff-24da2f4dadca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '321f21add6a240a09f41cd78713a15be', 'request_id': 'req-9645b4f4-8cb8-4f50-8dff-24da2f4dadca'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:27.686982 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9645b4f4-8cb8-4f50-8dff-24da2f4dadca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '321f21add6a240a09f41cd78713a15be', 'request_id': 'req-9645b4f4-8cb8-4f50-8dff-24da2f4dadca'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:27.757221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9645b4f4-8cb8-4f50-8dff-24da2f4dadca admin admin] The token cache is being invalidate because project 321f21add6a240a09f41cd78713a15be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:30:27.765163 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9645b4f4-8cb8-4f50-8dff-24da2f4dadca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:27.772148 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-9645b4f4-8cb8-4f50-8dff-24da2f4dadca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:27.778570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 827/1670] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:30:27 2022] DELETE /identity/v3/projects/321f21add6a240a09f41cd78713a15be => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:30:27.789114 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93649eef-c118-4b8a-b4ed-72fd71e46175 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:27.789401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93649eef-c118-4b8a-b4ed-72fd71e46175 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:27.789674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93649eef-c118-4b8a-b4ed-72fd71e46175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:27.791469 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69823c-2d23-49b3-a6eb-21e4bf5bded3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:27.792760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69823c-2d23-49b3-a6eb-21e4bf5bded3 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:27.793895 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69823c-2d23-49b3-a6eb-21e4bf5bded3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:27.794417 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69823c-2d23-49b3-a6eb-21e4bf5bded3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:27.796641 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-af69823c-2d23-49b3-a6eb-21e4bf5bded3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:27.991355 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-93649eef-c118-4b8a-b4ed-72fd71e46175 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:27.992661 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-93649eef-c118-4b8a-b4ed-72fd71e46175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:28.030060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af69823c-2d23-49b3-a6eb-21e4bf5bded3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:28.031091 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af69823c-2d23-49b3-a6eb-21e4bf5bded3 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:28.056391 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 828/1671] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:28.064180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 844/1672] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 889 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:28.075909 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d428de5-3d9a-48b6-99ad-e52c67d7d746 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:28.076431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d428de5-3d9a-48b6-99ad-e52c67d7d746 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:28.078748 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d428de5-3d9a-48b6-99ad-e52c67d7d746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:28.206963 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-0d428de5-3d9a-48b6-99ad-e52c67d7d746 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:28.215808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0d428de5-3d9a-48b6-99ad-e52c67d7d746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:28.289206 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 829/1673] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:28.292216 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69823c-2d23-49b3-a6eb-21e4bf5bded3 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:28.307491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69823c-2d23-49b3-a6eb-21e4bf5bded3 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:28.329888 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a6929-170d-4033-b4c9-eb24959057f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:28.336609 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a6929-170d-4033-b4c9-eb24959057f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:28.336609 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a6929-170d-4033-b4c9-eb24959057f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:28.336609 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a6929-170d-4033-b4c9-eb24959057f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:28.346552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a6929-170d-4033-b4c9-eb24959057f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:30:28.349181 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a6929-170d-4033-b4c9-eb24959057f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:28.356400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a6929-170d-4033-b4c9-eb24959057f7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:28.471362 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 845/1674] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:30:28.472858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d428de5-3d9a-48b6-99ad-e52c67d7d746 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:28.487898 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d428de5-3d9a-48b6-99ad-e52c67d7d746 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:28.518738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d10a68-56a9-4009-ad36-ee66efbbbb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:28.519381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d10a68-56a9-4009-ad36-ee66efbbbb7d admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:28.519658 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d10a68-56a9-4009-ad36-ee66efbbbb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:28.519968 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d10a68-56a9-4009-ad36-ee66efbbbb7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:28.533088 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d10a68-56a9-4009-ad36-ee66efbbbb7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:28.537609 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d10a68-56a9-4009-ad36-ee66efbbbb7d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:28.684211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a6929-170d-4033-b4c9-eb24959057f7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:28.696993 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 830/1675] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:28 2022] POST /identity/v3/projects => generated 383 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:28.715217 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a6929-170d-4033-b4c9-eb24959057f7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:28.769512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d10a68-56a9-4009-ad36-ee66efbbbb7d admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:28.771743 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b6d10a68-56a9-4009-ad36-ee66efbbbb7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:28.967238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:29.122765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:29.127382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5958e4-dcea-4a49-a8d5-bb797613a0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:29.134506 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5958e4-dcea-4a49-a8d5-bb797613a0b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:29.137286 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d10a68-56a9-4009-ad36-ee66efbbbb7d admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:29.141534 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5958e4-dcea-4a49-a8d5-bb797613a0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:29.157073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5958e4-dcea-4a49-a8d5-bb797613a0b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:29.157374 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d921e32-7ad8-421f-b51d-6d6fc371d798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:29.157374 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d921e32-7ad8-421f-b51d-6d6fc371d798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:29.157374 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d921e32-7ad8-421f-b51d-6d6fc371d798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:29.157374 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d921e32-7ad8-421f-b51d-6d6fc371d798 admin admin] PATH_INFO: `/v3/projects/1b036d583edb46fcb6a545db4952e6b3` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:29.157713 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d921e32-7ad8-421f-b51d-6d6fc371d798 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b036d583edb46fcb6a545db4952e6b3)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:29.157713 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d921e32-7ad8-421f-b51d-6d6fc371d798 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:29.157868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5958e4-dcea-4a49-a8d5-bb797613a0b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:29.162825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5958e4-dcea-4a49-a8d5-bb797613a0b1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:29.342688 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 846/1676] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:28 2022] POST /identity/v3/users => generated 292 bytes in 659 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:29.343874 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5958e4-dcea-4a49-a8d5-bb797613a0b1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:29.367802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-0d921e32-7ad8-421f-b51d-6d6fc371d798 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b036d583edb46fcb6a545db4952e6b3', 'request_id': 'req-0d921e32-7ad8-421f-b51d-6d6fc371d798'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:29.400973 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-0d921e32-7ad8-421f-b51d-6d6fc371d798 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b036d583edb46fcb6a545db4952e6b3', 'request_id': 'req-0d921e32-7ad8-421f-b51d-6d6fc371d798'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:29.403285 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5958e4-dcea-4a49-a8d5-bb797613a0b1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:29.434445 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-0d921e32-7ad8-421f-b51d-6d6fc371d798 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b036d583edb46fcb6a545db4952e6b3', 'request_id': 'req-0d921e32-7ad8-421f-b51d-6d6fc371d798'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:29.498900 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-0d921e32-7ad8-421f-b51d-6d6fc371d798 admin admin] The token cache is being invalidate because project 1b036d583edb46fcb6a545db4952e6b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:30:29.499522 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-0d921e32-7ad8-421f-b51d-6d6fc371d798 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:29.504317 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-0d921e32-7ad8-421f-b51d-6d6fc371d798 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:29.517306 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 831/1677] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:30:28 2022] DELETE /identity/v3/projects/1b036d583edb46fcb6a545db4952e6b3 => generated 0 bytes in 748 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:29.521182 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d921e32-7ad8-421f-b51d-6d6fc371d798 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:29.529319 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0d921e32-7ad8-421f-b51d-6d6fc371d798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:29.820256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61a5b7-8937-461d-844f-4c3c97138cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:29.826052 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61a5b7-8937-461d-844f-4c3c97138cfe admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:29.829489 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61a5b7-8937-461d-844f-4c3c97138cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:29.833191 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61a5b7-8937-461d-844f-4c3c97138cfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:29.841791 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61a5b7-8937-461d-844f-4c3c97138cfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:29.880256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61a5b7-8937-461d-844f-4c3c97138cfe admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:29.937640 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 847/1678] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 594 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:29.951583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61a5b7-8937-461d-844f-4c3c97138cfe admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:29.961562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8f61a5b7-8937-461d-844f-4c3c97138cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:30.017519 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d921e32-7ad8-421f-b51d-6d6fc371d798 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:30.043383 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61a5b7-8937-461d-844f-4c3c97138cfe admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:30.047140 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44410a4-03e9-49e0-95ab-b673952a5c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:30.047882 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44410a4-03e9-49e0-95ab-b673952a5c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:30.048173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44410a4-03e9-49e0-95ab-b673952a5c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:30.048504 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44410a4-03e9-49e0-95ab-b673952a5c9d admin admin] PATH_INFO: `/v3/users/e7ac077f28514e85b9adcafb04e0dc0a` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:30.063146 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44410a4-03e9-49e0-95ab-b673952a5c9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7ac077f28514e85b9adcafb04e0dc0a)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:30.065749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44410a4-03e9-49e0-95ab-b673952a5c9d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:30.080355 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b624d1-05e1-4ab0-a5d6-e969e8893da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:30.080999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b624d1-05e1-4ab0-a5d6-e969e8893da9 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:30.081280 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b624d1-05e1-4ab0-a5d6-e969e8893da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:30.081566 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b624d1-05e1-4ab0-a5d6-e969e8893da9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:30.085016 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b624d1-05e1-4ab0-a5d6-e969e8893da9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:30.091431 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b624d1-05e1-4ab0-a5d6-e969e8893da9 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:30.156562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 848/1679] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:30.166271 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b624d1-05e1-4ab0-a5d6-e969e8893da9 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:30.219401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b624d1-05e1-4ab0-a5d6-e969e8893da9 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:30.234565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973eedcb-f9e6-47e4-9e0e-92fa537dda6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:30.235324 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eedcb-f9e6-47e4-9e0e-92fa537dda6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:30.235882 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eedcb-f9e6-47e4-9e0e-92fa537dda6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:30.236274 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973eedcb-f9e6-47e4-9e0e-92fa537dda6f admin admin] PATH_INFO: `/v3/projects/abd205f7acc3405e872cca67f4549aae/users/695e2b08794a413daaa9e51de5a650af/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:30.306007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973eedcb-f9e6-47e4-9e0e-92fa537dda6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=abd205f7acc3405e872cca67f4549aae, user_id=695e2b08794a413daaa9e51de5a650af, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:30.311258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973eedcb-f9e6-47e4-9e0e-92fa537dda6f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:30.326014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d44410a4-03e9-49e0-95ab-b673952a5c9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7ac077f28514e85b9adcafb04e0dc0a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:30.343558 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d44410a4-03e9-49e0-95ab-b673952a5c9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7ac077f28514e85b9adcafb04e0dc0a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:30.347715 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 849/1680] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:30 2022] PUT /identity/v3/projects/abd205f7acc3405e872cca67f4549aae/users/695e2b08794a413daaa9e51de5a650af/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:30.364761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95cc9a-08aa-48c1-a4f3-94cc09642f8e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:30.370639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95cc9a-08aa-48c1-a4f3-94cc09642f8e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:30.370948 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95cc9a-08aa-48c1-a4f3-94cc09642f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:30.403441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d44410a4-03e9-49e0-95ab-b673952a5c9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7ac077f28514e85b9adcafb04e0dc0a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:30.406400 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d44410a4-03e9-49e0-95ab-b673952a5c9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7ac077f28514e85b9adcafb04e0dc0a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:30.410100 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d44410a4-03e9-49e0-95ab-b673952a5c9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7ac077f28514e85b9adcafb04e0dc0a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:30.453804 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 832/1681] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:30:29 2022] DELETE /identity/v3/users/e7ac077f28514e85b9adcafb04e0dc0a => generated 0 bytes in 933 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:30.544337 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8a95cc9a-08aa-48c1-a4f3-94cc09642f8e None None] MFA Rules not processed for user `695e2b08794a413daaa9e51de5a650af`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:30.549023 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8a95cc9a-08aa-48c1-a4f3-94cc09642f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:30.827052 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 850/1682] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:30.832449 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44410a4-03e9-49e0-95ab-b673952a5c9d admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:30.835049 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d44410a4-03e9-49e0-95ab-b673952a5c9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:30.856802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:31.075746 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44410a4-03e9-49e0-95ab-b673952a5c9d admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:31.086652 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b43bc6-78cb-4ea9-960a-36f1a9b3097e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:31.090366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b43bc6-78cb-4ea9-960a-36f1a9b3097e admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:31.090994 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b43bc6-78cb-4ea9-960a-36f1a9b3097e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:31.091384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b43bc6-78cb-4ea9-960a-36f1a9b3097e admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:31.092219 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b43bc6-78cb-4ea9-960a-36f1a9b3097e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:31.096876 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b43bc6-78cb-4ea9-960a-36f1a9b3097e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:31.165022 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 833/1683] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:30 2022] POST /identity/v3/projects => generated 383 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:31.172214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a95cc9a-08aa-48c1-a4f3-94cc09642f8e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:31.196330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a95cc9a-08aa-48c1-a4f3-94cc09642f8e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:31.270737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:31.464735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab124b-622f-482e-af13-1c4fa74be9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:31.465310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab124b-622f-482e-af13-1c4fa74be9b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:31.465590 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab124b-622f-482e-af13-1c4fa74be9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:31.465870 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab124b-622f-482e-af13-1c4fa74be9b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:31.470463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab124b-622f-482e-af13-1c4fa74be9b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:31.473838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab124b-622f-482e-af13-1c4fa74be9b0 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:31.547826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 851/1684] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:31 2022] POST /identity/v3/users => generated 298 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:31.550077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b43bc6-78cb-4ea9-960a-36f1a9b3097e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:31.599994 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b43bc6-78cb-4ea9-960a-36f1a9b3097e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:31.637293 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100a974-a612-462d-9325-e663a3bde882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:31.638999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100a974-a612-462d-9325-e663a3bde882 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:31.640918 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100a974-a612-462d-9325-e663a3bde882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:31.642077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100a974-a612-462d-9325-e663a3bde882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:31.647269 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e100a974-a612-462d-9325-e663a3bde882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:31.649836 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e100a974-a612-462d-9325-e663a3bde882 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:31.703297 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 834/1685] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:31.709183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab124b-622f-482e-af13-1c4fa74be9b0 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:31.739866 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100a974-a612-462d-9325-e663a3bde882 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:31.759752 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e100a974-a612-462d-9325-e663a3bde882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:31.763064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab124b-622f-482e-af13-1c4fa74be9b0 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:31.791600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45d939-de4e-48bc-9d7f-ce2e2177999d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:31.792612 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45d939-de4e-48bc-9d7f-ce2e2177999d admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:31.792902 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45d939-de4e-48bc-9d7f-ce2e2177999d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:31.793179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45d939-de4e-48bc-9d7f-ce2e2177999d admin admin] PATH_INFO: `/v3/projects/49da3e9e24d64d01b7d343a562830e31/users/e107c439e2c8451082ee1e7a8d418fe5/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:31.804935 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100a974-a612-462d-9325-e663a3bde882 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:31.843494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d302fbbf-180a-47e9-b26c-dfc83babfd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:31.844915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302fbbf-180a-47e9-b26c-dfc83babfd69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:31.846233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302fbbf-180a-47e9-b26c-dfc83babfd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:31.846936 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302fbbf-180a-47e9-b26c-dfc83babfd69 admin admin] PATH_INFO: `/v3/projects/923d72d5b3294f65aa6fdb0203731a27` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:31.856501 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d302fbbf-180a-47e9-b26c-dfc83babfd69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=923d72d5b3294f65aa6fdb0203731a27)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:31.861369 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d302fbbf-180a-47e9-b26c-dfc83babfd69 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:31.880963 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45d939-de4e-48bc-9d7f-ce2e2177999d admin admin] RBAC: Authorizing `identity:create_grant(project_id=49da3e9e24d64d01b7d343a562830e31, user_id=e107c439e2c8451082ee1e7a8d418fe5, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:31.889117 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45d939-de4e-48bc-9d7f-ce2e2177999d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:31.991335 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 852/1686] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:31 2022] PUT /identity/v3/projects/49da3e9e24d64d01b7d343a562830e31/users/e107c439e2c8451082ee1e7a8d418fe5/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:31.996114 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45d939-de4e-48bc-9d7f-ce2e2177999d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:32.035032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d302fbbf-180a-47e9-b26c-dfc83babfd69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '923d72d5b3294f65aa6fdb0203731a27', 'request_id': 'req-d302fbbf-180a-47e9-b26c-dfc83babfd69'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:32.065116 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d302fbbf-180a-47e9-b26c-dfc83babfd69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '923d72d5b3294f65aa6fdb0203731a27', 'request_id': 'req-d302fbbf-180a-47e9-b26c-dfc83babfd69'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:32.084272 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d302fbbf-180a-47e9-b26c-dfc83babfd69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '923d72d5b3294f65aa6fdb0203731a27', 'request_id': 'req-d302fbbf-180a-47e9-b26c-dfc83babfd69'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:32.163515 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d302fbbf-180a-47e9-b26c-dfc83babfd69 admin admin] The token cache is being invalidate because project 923d72d5b3294f65aa6fdb0203731a27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:30:32.163818 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d302fbbf-180a-47e9-b26c-dfc83babfd69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:32.168691 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d302fbbf-180a-47e9-b26c-dfc83babfd69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:32.170389 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 835/1687] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:30:31 2022] DELETE /identity/v3/projects/923d72d5b3294f65aa6fdb0203731a27 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:32.181306 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3799e0e-1130-4e7e-bc2e-ab4d71b5dad1 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:32.181587 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3799e0e-1130-4e7e-bc2e-ab4d71b5dad1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:32.181867 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3799e0e-1130-4e7e-bc2e-ab4d71b5dad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:32.274949 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-a3799e0e-1130-4e7e-bc2e-ab4d71b5dad1 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:32.276187 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a3799e0e-1130-4e7e-bc2e-ab4d71b5dad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:32.295743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45d939-de4e-48bc-9d7f-ce2e2177999d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:32.515925 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 836/1688] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:32.518249 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa0654-81d6-47f0-85b2-4060ea49c6f6 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:32.518540 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa0654-81d6-47f0-85b2-4060ea49c6f6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:32.518865 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa0654-81d6-47f0-85b2-4060ea49c6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:32.538852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:32.541726 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2963d-a0c3-4677-8be3-568de3db3056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:32.544509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2963d-a0c3-4677-8be3-568de3db3056 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:32.545660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2963d-a0c3-4677-8be3-568de3db3056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:32.546917 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2963d-a0c3-4677-8be3-568de3db3056 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:32.547799 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2963d-a0c3-4677-8be3-568de3db3056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:32.556330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2963d-a0c3-4677-8be3-568de3db3056 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:32.597617 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-84fa0654-81d6-47f0-85b2-4060ea49c6f6 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:32.600445 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 853/1689] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 607 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 12:30:32.604441 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2963d-a0c3-4677-8be3-568de3db3056 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:32.605972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-84fa0654-81d6-47f0-85b2-4060ea49c6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:32.624474 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-27c2963d-a0c3-4677-8be3-568de3db3056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:32.671352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 837/1690] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:30:32.673803 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2842a-ac0c-4aaa-9c13-5205ff5f95c2 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:32.674093 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2842a-ac0c-4aaa-9c13-5205ff5f95c2 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:32.674372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2842a-ac0c-4aaa-9c13-5205ff5f95c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:32.706100 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2963d-a0c3-4677-8be3-568de3db3056 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:32.724369 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f6a4a-026f-4027-b581-1a306b3002b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:32.725071 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f6a4a-026f-4027-b581-1a306b3002b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:32.725378 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f6a4a-026f-4027-b581-1a306b3002b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:32.725699 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f6a4a-026f-4027-b581-1a306b3002b4 admin admin] PATH_INFO: `/v3/projects/49da3e9e24d64d01b7d343a562830e31/users/e107c439e2c8451082ee1e7a8d418fe5/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:32.745935 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f6a4a-026f-4027-b581-1a306b3002b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49da3e9e24d64d01b7d343a562830e31, user_id=e107c439e2c8451082ee1e7a8d418fe5, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:32.753166 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f6a4a-026f-4027-b581-1a306b3002b4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:32.785899 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-72d2842a-ac0c-4aaa-9c13-5205ff5f95c2 None None] MFA Rules not processed for user `55563aaa662246c5aaf107145475bd76`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:32.790065 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-72d2842a-ac0c-4aaa-9c13-5205ff5f95c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:32.799917 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 854/1691] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:32 2022] PUT /identity/v3/projects/49da3e9e24d64d01b7d343a562830e31/users/e107c439e2c8451082ee1e7a8d418fe5/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:32.804776 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f6a4a-026f-4027-b581-1a306b3002b4 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:32.874293 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:32.973884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 838/1692] 10.222.0.27 () {58 vars in 1015 bytes} [Sat Aug 6 12:30:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:32.975055 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9266c-6058-49a0-bbf5-5a5a58ea4384 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:32.975359 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9266c-6058-49a0-bbf5-5a5a58ea4384 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:32.975599 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9266c-6058-49a0-bbf5-5a5a58ea4384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:33.035243 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f6a4a-026f-4027-b581-1a306b3002b4 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:33.047613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e15809-1b29-469c-b73a-3084d9049ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:33.048561 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e15809-1b29-469c-b73a-3084d9049ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:33.049052 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e15809-1b29-469c-b73a-3084d9049ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:33.051962 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e15809-1b29-469c-b73a-3084d9049ee6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:33.053422 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e15809-1b29-469c-b73a-3084d9049ee6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:30:33.053867 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e15809-1b29-469c-b73a-3084d9049ee6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:33.055814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e15809-1b29-469c-b73a-3084d9049ee6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:33.081849 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-98c9266c-6058-49a0-bbf5-5a5a58ea4384 None None] MFA Rules not processed for user `e107c439e2c8451082ee1e7a8d418fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:33.084690 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-98c9266c-6058-49a0-bbf5-5a5a58ea4384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:33.097297 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 855/1693] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:30:33.098657 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e15809-1b29-469c-b73a-3084d9049ee6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:33.135389 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e15809-1b29-469c-b73a-3084d9049ee6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:33.151982 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53303e7d-5968-4c16-88be-6b400cde3fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:33.162151 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53303e7d-5968-4c16-88be-6b400cde3fa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:33.162151 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53303e7d-5968-4c16-88be-6b400cde3fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:33.162457 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53303e7d-5968-4c16-88be-6b400cde3fa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:33.163013 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53303e7d-5968-4c16-88be-6b400cde3fa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:33.171665 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53303e7d-5968-4c16-88be-6b400cde3fa5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:33.241862 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 856/1694] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:33 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:33.249913 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53303e7d-5968-4c16-88be-6b400cde3fa5 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:33.438441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 839/1695] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:32 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:33.465430 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9266c-6058-49a0-bbf5-5a5a58ea4384 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:33.467987 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-98c9266c-6058-49a0-bbf5-5a5a58ea4384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:33.576324 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53303e7d-5968-4c16-88be-6b400cde3fa5 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:33.618672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b039d-2854-4dd3-bec8-1ef6009ebdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:33.620733 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b039d-2854-4dd3-bec8-1ef6009ebdfc admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:33.621013 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b039d-2854-4dd3-bec8-1ef6009ebdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:33.621287 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b039d-2854-4dd3-bec8-1ef6009ebdfc admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:33.642547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3b039d-2854-4dd3-bec8-1ef6009ebdfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:33.651703 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3b039d-2854-4dd3-bec8-1ef6009ebdfc admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:33.731200 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 857/1696] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:33 2022] POST /identity/v3/users => generated 287 bytes in 488 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:33.734067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b039d-2854-4dd3-bec8-1ef6009ebdfc admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:33.738606 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3f3b039d-2854-4dd3-bec8-1ef6009ebdfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:33.762165 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9266c-6058-49a0-bbf5-5a5a58ea4384 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:33.900095 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b039d-2854-4dd3-bec8-1ef6009ebdfc admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:33.909005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c1fc2-7a91-4b80-947d-7253c7d2c3c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:33.911564 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:33.913023 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c1fc2-7a91-4b80-947d-7253c7d2c3c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:33.913325 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c1fc2-7a91-4b80-947d-7253c7d2c3c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:33.913596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c1fc2-7a91-4b80-947d-7253c7d2c3c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:33.920553 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4dc124-0357-428f-8aa7-6b1fd63dd414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:33.921389 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dc124-0357-428f-8aa7-6b1fd63dd414 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:33.923277 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dc124-0357-428f-8aa7-6b1fd63dd414 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:33.923711 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dc124-0357-428f-8aa7-6b1fd63dd414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:33.945993 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c1fc2-7a91-4b80-947d-7253c7d2c3c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:33.947442 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c1fc2-7a91-4b80-947d-7253c7d2c3c2 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:33.983944 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 858/1697] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:33.984946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c1fc2-7a91-4b80-947d-7253c7d2c3c2 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:34.033149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c1fc2-7a91-4b80-947d-7253c7d2c3c2 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:34.051035 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7927f-7cfc-440a-ba39-2a3f0ef25bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:34.053347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7927f-7cfc-440a-ba39-2a3f0ef25bdf admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:34.053347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7927f-7cfc-440a-ba39-2a3f0ef25bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:34.053717 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7927f-7cfc-440a-ba39-2a3f0ef25bdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:34.054696 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b7927f-7cfc-440a-ba39-2a3f0ef25bdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:34.055935 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b7927f-7cfc-440a-ba39-2a3f0ef25bdf admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:34.093843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 859/1698] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:34.094213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7927f-7cfc-440a-ba39-2a3f0ef25bdf admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:34.103298 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f4b7927f-7cfc-440a-ba39-2a3f0ef25bdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:34.112749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4dc124-0357-428f-8aa7-6b1fd63dd414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:34.116864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4dc124-0357-428f-8aa7-6b1fd63dd414 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:34.137472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7927f-7cfc-440a-ba39-2a3f0ef25bdf admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:34.156431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 840/1699] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:33 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 691 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:34.157175 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3411a02d-e57d-461f-b25e-080cc29656a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:34.157819 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411a02d-e57d-461f-b25e-080cc29656a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:34.158133 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411a02d-e57d-461f-b25e-080cc29656a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:34.158375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411a02d-e57d-461f-b25e-080cc29656a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:34.164223 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4dc124-0357-428f-8aa7-6b1fd63dd414 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:34.166536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:34.174543 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3411a02d-e57d-461f-b25e-080cc29656a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:34.175670 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3411a02d-e57d-461f-b25e-080cc29656a3 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:34.210972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4dc124-0357-428f-8aa7-6b1fd63dd414 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:34.220547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 860/1700] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:34.227264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa142aa-2c7a-42c1-91cd-666ad3dbaf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:34.232685 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa142aa-2c7a-42c1-91cd-666ad3dbaf66 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:34.236509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa142aa-2c7a-42c1-91cd-666ad3dbaf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:34.236881 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa142aa-2c7a-42c1-91cd-666ad3dbaf66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:34.238356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa142aa-2c7a-42c1-91cd-666ad3dbaf66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:34.241188 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa142aa-2c7a-42c1-91cd-666ad3dbaf66 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:34.286956 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 841/1701] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:34.288988 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3411a02d-e57d-461f-b25e-080cc29656a3 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:34.319548 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3411a02d-e57d-461f-b25e-080cc29656a3 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:34.347800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be00068-7693-40eb-9bf3-177fff7b09bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:34.348578 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be00068-7693-40eb-9bf3-177fff7b09bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:34.348864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be00068-7693-40eb-9bf3-177fff7b09bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:34.349141 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be00068-7693-40eb-9bf3-177fff7b09bf admin admin] PATH_INFO: `/v3/projects/386778dcad4d45c7bc5c343b735983b2/users/1a19e414e5ab48fb8fd2615e694dec41/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:34.372426 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa142aa-2c7a-42c1-91cd-666ad3dbaf66 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:34.387269 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa142aa-2c7a-42c1-91cd-666ad3dbaf66 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:34.399633 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80431e54-81e5-4c86-9610-eac509a24489 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:34.402266 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80431e54-81e5-4c86-9610-eac509a24489 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:34.402997 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80431e54-81e5-4c86-9610-eac509a24489 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:34.406127 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80431e54-81e5-4c86-9610-eac509a24489 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:34.410049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be00068-7693-40eb-9bf3-177fff7b09bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=386778dcad4d45c7bc5c343b735983b2, user_id=1a19e414e5ab48fb8fd2615e694dec41, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:34.417733 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-80431e54-81e5-4c86-9610-eac509a24489 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:34.436124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be00068-7693-40eb-9bf3-177fff7b09bf admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:34.469119 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 861/1702] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:34 2022] PUT /identity/v3/projects/386778dcad4d45c7bc5c343b735983b2/users/1a19e414e5ab48fb8fd2615e694dec41/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:34.489803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5174a7-d42a-4ea1-85d3-74d9c2f4ac08 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:34.489803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5174a7-d42a-4ea1-85d3-74d9c2f4ac08 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:34.489803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5174a7-d42a-4ea1-85d3-74d9c2f4ac08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:34.573969 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-eb5174a7-d42a-4ea1-85d3-74d9c2f4ac08 None None] MFA Rules not processed for user `1a19e414e5ab48fb8fd2615e694dec41`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:34.577481 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-eb5174a7-d42a-4ea1-85d3-74d9c2f4ac08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:34.581770 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80431e54-81e5-4c86-9610-eac509a24489 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:34.595910 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80431e54-81e5-4c86-9610-eac509a24489 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:34.639222 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 842/1703] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:30:34.801826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 862/1704] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:35.317664 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80431e54-81e5-4c86-9610-eac509a24489 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:35.336543 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80431e54-81e5-4c86-9610-eac509a24489 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:35.603978 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c83f4-83b5-4fcb-9f44-3069d62f0e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:35.605271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c83f4-83b5-4fcb-9f44-3069d62f0e66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:35.605717 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c83f4-83b5-4fcb-9f44-3069d62f0e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:35.606193 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c83f4-83b5-4fcb-9f44-3069d62f0e66 admin admin] PATH_INFO: `/v3/users/695e2b08794a413daaa9e51de5a650af` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:35.619317 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c83f4-83b5-4fcb-9f44-3069d62f0e66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=695e2b08794a413daaa9e51de5a650af)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:35.624158 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c83f4-83b5-4fcb-9f44-3069d62f0e66 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:35.707734 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:35.753168 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-1c6c83f4-83b5-4fcb-9f44-3069d62f0e66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '695e2b08794a413daaa9e51de5a650af'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:35.775632 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-1c6c83f4-83b5-4fcb-9f44-3069d62f0e66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '695e2b08794a413daaa9e51de5a650af'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:35.818321 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-1c6c83f4-83b5-4fcb-9f44-3069d62f0e66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '695e2b08794a413daaa9e51de5a650af'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:35.821190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-1c6c83f4-83b5-4fcb-9f44-3069d62f0e66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '695e2b08794a413daaa9e51de5a650af'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:35.826139 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-1c6c83f4-83b5-4fcb-9f44-3069d62f0e66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '695e2b08794a413daaa9e51de5a650af'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:35.846616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5174a7-d42a-4ea1-85d3-74d9c2f4ac08 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:35.849144 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-eb5174a7-d42a-4ea1-85d3-74d9c2f4ac08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:35.867791 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 843/1705] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:30:35 2022] DELETE /identity/v3/users/695e2b08794a413daaa9e51de5a650af => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:35.874536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da9e22-56d4-4903-9a08-d9198243c791 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:35.875074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da9e22-56d4-4903-9a08-d9198243c791 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:35.879074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da9e22-56d4-4903-9a08-d9198243c791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:35.914951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-a0da9e22-56d4-4903-9a08-d9198243c791 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:35.916785 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a0da9e22-56d4-4903-9a08-d9198243c791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:36.126915 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5174a7-d42a-4ea1-85d3-74d9c2f4ac08 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:36.171644 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 844/1706] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:30:36.245626 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da9e22-56d4-4903-9a08-d9198243c791 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:36.248328 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a0da9e22-56d4-4903-9a08-d9198243c791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:36.301781 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:36.349377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77d593-314c-43e4-93cc-4d6c7ef93049 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:36.352460 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77d593-314c-43e4-93cc-4d6c7ef93049 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:36.353742 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77d593-314c-43e4-93cc-4d6c7ef93049 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:36.354248 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77d593-314c-43e4-93cc-4d6c7ef93049 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:36.358707 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9e77d593-314c-43e4-93cc-4d6c7ef93049 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:36.574350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da9e22-56d4-4903-9a08-d9198243c791 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:36.596083 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0298cd97-fa94-4377-91b9-faee153a435a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:36.598514 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298cd97-fa94-4377-91b9-faee153a435a service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:36.600923 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298cd97-fa94-4377-91b9-faee153a435a service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:36.601588 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298cd97-fa94-4377-91b9-faee153a435a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:36.619310 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0298cd97-fa94-4377-91b9-faee153a435a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:36.622377 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0298cd97-fa94-4377-91b9-faee153a435a service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:36.658417 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 845/1707] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:36.707157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77d593-314c-43e4-93cc-4d6c7ef93049 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:36.719362 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77d593-314c-43e4-93cc-4d6c7ef93049 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:36.782775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 863/1708] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 940 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:30:37.244229 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0298cd97-fa94-4377-91b9-faee153a435a service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:37.247755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0298cd97-fa94-4377-91b9-faee153a435a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:37.287229 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0298cd97-fa94-4377-91b9-faee153a435a service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:37.294123 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8580e7-0a14-4176-9600-c0ca3a9cee4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:37.295137 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8580e7-0a14-4176-9600-c0ca3a9cee4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:37.295534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8580e7-0a14-4176-9600-c0ca3a9cee4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:37.295955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8580e7-0a14-4176-9600-c0ca3a9cee4a admin admin] PATH_INFO: `/v3/projects/f360267e7953431e87099a0c030e55a7` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:37.297619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8580e7-0a14-4176-9600-c0ca3a9cee4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f360267e7953431e87099a0c030e55a7)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:37.300227 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8580e7-0a14-4176-9600-c0ca3a9cee4a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:37.393438 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:37.406595 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4a8580e7-0a14-4176-9600-c0ca3a9cee4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f360267e7953431e87099a0c030e55a7', 'request_id': 'req-4a8580e7-0a14-4176-9600-c0ca3a9cee4a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:37.430088 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4a8580e7-0a14-4176-9600-c0ca3a9cee4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f360267e7953431e87099a0c030e55a7', 'request_id': 'req-4a8580e7-0a14-4176-9600-c0ca3a9cee4a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:37.449127 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4a8580e7-0a14-4176-9600-c0ca3a9cee4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f360267e7953431e87099a0c030e55a7', 'request_id': 'req-4a8580e7-0a14-4176-9600-c0ca3a9cee4a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:37.468873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77d593-314c-43e4-93cc-4d6c7ef93049 service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:37.474994 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9e77d593-314c-43e4-93cc-4d6c7ef93049 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:37.491949 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4a8580e7-0a14-4176-9600-c0ca3a9cee4a admin admin] The token cache is being invalidate because project f360267e7953431e87099a0c030e55a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:30:37.492684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4a8580e7-0a14-4176-9600-c0ca3a9cee4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:37.495141 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4a8580e7-0a14-4176-9600-c0ca3a9cee4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:37.501583 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 846/1709] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:30:37 2022] DELETE /identity/v3/projects/f360267e7953431e87099a0c030e55a7 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:37.507973 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54b377-7a61-49e6-b9fc-fb5ae6f140a1 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:37.508803 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54b377-7a61-49e6-b9fc-fb5ae6f140a1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:37.509189 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54b377-7a61-49e6-b9fc-fb5ae6f140a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:37.559670 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77d593-314c-43e4-93cc-4d6c7ef93049 service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:37.572530 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ae54b377-7a61-49e6-b9fc-fb5ae6f140a1 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:37.575539 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ae54b377-7a61-49e6-b9fc-fb5ae6f140a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:37.818637 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 847/1710] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:37.828082 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ac885-98c1-4deb-82fd-8a39834bcb8f None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:37.831006 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ac885-98c1-4deb-82fd-8a39834bcb8f None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:37.832832 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ac885-98c1-4deb-82fd-8a39834bcb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:37.836407 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d970068-d6e3-4ef6-9a9f-b355ce318c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:37.837139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d970068-d6e3-4ef6-9a9f-b355ce318c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:37.842731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d970068-d6e3-4ef6-9a9f-b355ce318c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:37.843030 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d970068-d6e3-4ef6-9a9f-b355ce318c54 admin admin] PATH_INFO: `/v3/users/8c5ea1856efd4c0c9b4f615e1811b951` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:37.847151 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d970068-d6e3-4ef6-9a9f-b355ce318c54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c5ea1856efd4c0c9b4f615e1811b951)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:37.853320 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d970068-d6e3-4ef6-9a9f-b355ce318c54 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:37.886219 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-e24ac885-98c1-4deb-82fd-8a39834bcb8f None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:37.889550 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e24ac885-98c1-4deb-82fd-8a39834bcb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:37.903002 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 848/1711] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:30:37.906912 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ac885-98c1-4deb-82fd-8a39834bcb8f None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:37.951144 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ac885-98c1-4deb-82fd-8a39834bcb8f None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:37.966506 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f4e59-0985-40e6-86f4-26949a586292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:37.968435 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f4e59-0985-40e6-86f4-26949a586292 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:37.969833 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f4e59-0985-40e6-86f4-26949a586292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:37.970803 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f4e59-0985-40e6-86f4-26949a586292 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:37.972736 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f4e59-0985-40e6-86f4-26949a586292 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:30:37.974246 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f4e59-0985-40e6-86f4-26949a586292 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:37.978266 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f4e59-0985-40e6-86f4-26949a586292 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:38.013878 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 849/1712] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:30:38.016611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6d970068-d6e3-4ef6-9a9f-b355ce318c54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c5ea1856efd4c0c9b4f615e1811b951'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:38.027410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f4e59-0985-40e6-86f4-26949a586292 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:38.057356 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6d970068-d6e3-4ef6-9a9f-b355ce318c54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c5ea1856efd4c0c9b4f615e1811b951'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:38.059816 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6d970068-d6e3-4ef6-9a9f-b355ce318c54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c5ea1856efd4c0c9b4f615e1811b951'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:38.061208 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6d970068-d6e3-4ef6-9a9f-b355ce318c54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c5ea1856efd4c0c9b4f615e1811b951'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:38.075454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f4e59-0985-40e6-86f4-26949a586292 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:38.079711 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6d970068-d6e3-4ef6-9a9f-b355ce318c54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c5ea1856efd4c0c9b4f615e1811b951'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:38.116604 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610190e-46a9-4c39-95d9-820140b1d335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:38.117293 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610190e-46a9-4c39-95d9-820140b1d335 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:38.122767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 864/1713] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:30:37 2022] DELETE /identity/v3/users/8c5ea1856efd4c0c9b4f615e1811b951 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:30:38.124952 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610190e-46a9-4c39-95d9-820140b1d335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:38.125356 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108a15f-1ea2-4f87-b45f-5784e24c71d6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:38.125656 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108a15f-1ea2-4f87-b45f-5784e24c71d6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:38.125963 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610190e-46a9-4c39-95d9-820140b1d335 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:38.126959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610190e-46a9-4c39-95d9-820140b1d335 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:38.127562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108a15f-1ea2-4f87-b45f-5784e24c71d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:38.148263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3610190e-46a9-4c39-95d9-820140b1d335 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:38.202752 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f108a15f-1ea2-4f87-b45f-5784e24c71d6 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:38.204348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f108a15f-1ea2-4f87-b45f-5784e24c71d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:38.243371 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:38.285884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 850/1714] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:38 2022] POST /identity/v3/projects => generated 365 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:38.289929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610190e-46a9-4c39-95d9-820140b1d335 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:38.295786 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3610190e-46a9-4c39-95d9-820140b1d335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:38.420464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 865/1715] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:38.471512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610190e-46a9-4c39-95d9-820140b1d335 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:38.505674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb70ce-469c-4fb9-8ca5-8bb764991cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:38.507276 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb70ce-469c-4fb9-8ca5-8bb764991cad admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:38.508026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb70ce-469c-4fb9-8ca5-8bb764991cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:38.508974 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb70ce-469c-4fb9-8ca5-8bb764991cad admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:38.510075 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb70ce-469c-4fb9-8ca5-8bb764991cad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:38.521659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb70ce-469c-4fb9-8ca5-8bb764991cad admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:38.588770 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f108a15f-1ea2-4f87-b45f-5784e24c71d6 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:38.600888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 851/1716] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:38 2022] POST /identity/v3/users => generated 283 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:38.603879 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f108a15f-1ea2-4f87-b45f-5784e24c71d6 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:38.632620 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7af8e-6c2d-44cb-ab55-04672e6d5048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:38.640410 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7af8e-6c2d-44cb-ab55-04672e6d5048 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:38.640782 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7af8e-6c2d-44cb-ab55-04672e6d5048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:38.641024 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7af8e-6c2d-44cb-ab55-04672e6d5048 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:38.641797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7af8e-6c2d-44cb-ab55-04672e6d5048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:38.649355 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7af8e-6c2d-44cb-ab55-04672e6d5048 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:38.702461 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb70ce-469c-4fb9-8ca5-8bb764991cad admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:38.715247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 866/1717] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 12:30:38.721718 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb70ce-469c-4fb9-8ca5-8bb764991cad admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:38.740047 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c19076-6cda-4c0e-be5d-643f6390aedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:38.741184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c19076-6cda-4c0e-be5d-643f6390aedd admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:38.741440 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c19076-6cda-4c0e-be5d-643f6390aedd admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:38.741891 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c19076-6cda-4c0e-be5d-643f6390aedd admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:38.742853 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c19076-6cda-4c0e-be5d-643f6390aedd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:38.746681 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c19076-6cda-4c0e-be5d-643f6390aedd admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:38.781885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7af8e-6c2d-44cb-ab55-04672e6d5048 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:38.792523 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 852/1718] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:38.796271 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7af8e-6c2d-44cb-ab55-04672e6d5048 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:38.810259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae39da-dbc3-4e7b-b4fa-784df7f8abe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:38.811506 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae39da-dbc3-4e7b-b4fa-784df7f8abe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:38.814096 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae39da-dbc3-4e7b-b4fa-784df7f8abe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:38.814562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae39da-dbc3-4e7b-b4fa-784df7f8abe2 admin admin] PATH_INFO: `/v3/projects/dc9c1a0a10e74d6eb0bc048c0e032b73/users/5c14ab0d76364390b1690cfcee968791/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:38.849206 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae39da-dbc3-4e7b-b4fa-784df7f8abe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc9c1a0a10e74d6eb0bc048c0e032b73, user_id=5c14ab0d76364390b1690cfcee968791, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:38.852576 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae39da-dbc3-4e7b-b4fa-784df7f8abe2 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:38.905147 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 867/1719] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:38 2022] PUT /identity/v3/projects/dc9c1a0a10e74d6eb0bc048c0e032b73/users/5c14ab0d76364390b1690cfcee968791/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:38.930390 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd130125-83c1-44d2-ae1e-2d7250cd0400 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:38.930800 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd130125-83c1-44d2-ae1e-2d7250cd0400 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:38.931028 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd130125-83c1-44d2-ae1e-2d7250cd0400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:38.972546 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:39.024513 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-fd130125-83c1-44d2-ae1e-2d7250cd0400 None None] MFA Rules not processed for user `5c14ab0d76364390b1690cfcee968791`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:39.025820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fd130125-83c1-44d2-ae1e-2d7250cd0400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:39.307180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 853/1720] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:30:39.308147 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae39da-dbc3-4e7b-b4fa-784df7f8abe2 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:39.356360 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd130125-83c1-44d2-ae1e-2d7250cd0400 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:39.359368 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fd130125-83c1-44d2-ae1e-2d7250cd0400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:39.650485 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae39da-dbc3-4e7b-b4fa-784df7f8abe2 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:39.682746 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c577d5e3-33d5-4313-9a43-a08a30d41aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:39.684658 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577d5e3-33d5-4313-9a43-a08a30d41aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:39.686717 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577d5e3-33d5-4313-9a43-a08a30d41aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:39.687271 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577d5e3-33d5-4313-9a43-a08a30d41aa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:39.687763 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c577d5e3-33d5-4313-9a43-a08a30d41aa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:39.701144 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd130125-83c1-44d2-ae1e-2d7250cd0400 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:39.702937 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c577d5e3-33d5-4313-9a43-a08a30d41aa2 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:39.721570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9e3b7-0fd9-432b-a8f7-1dfd8f072d92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:39.722161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9e3b7-0fd9-432b-a8f7-1dfd8f072d92 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:39.722448 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9e3b7-0fd9-432b-a8f7-1dfd8f072d92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:39.722729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9e3b7-0fd9-432b-a8f7-1dfd8f072d92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:39.771504 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9e3b7-0fd9-432b-a8f7-1dfd8f072d92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:39.773005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9e3b7-0fd9-432b-a8f7-1dfd8f072d92 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:39.810755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 854/1721] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:30:39.819993 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 868/1722] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:39 2022] POST /identity/v3/projects => generated 363 bytes in 512 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 12:30:39.825931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9e3b7-0fd9-432b-a8f7-1dfd8f072d92 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:39.862422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9e3b7-0fd9-432b-a8f7-1dfd8f072d92 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:39.955726 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e1e80-8697-4e4f-80bc-bf091a67123b None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:39.956190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e1e80-8697-4e4f-80bc-bf091a67123b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:39.958894 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e1e80-8697-4e4f-80bc-bf091a67123b None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:39.960336 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 869/1723] 10.222.0.20 () {54 vars in 921 bytes} [Sat Aug 6 12:30:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:30:40.062058 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15742a1b-aa8b-4b3b-927f-8375c8d1b2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:40.063169 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15742a1b-aa8b-4b3b-927f-8375c8d1b2ea admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:40.063682 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15742a1b-aa8b-4b3b-927f-8375c8d1b2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:40.064111 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15742a1b-aa8b-4b3b-927f-8375c8d1b2ea admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:40.069726 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15742a1b-aa8b-4b3b-927f-8375c8d1b2ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:40.076123 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15742a1b-aa8b-4b3b-927f-8375c8d1b2ea admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:40.136511 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 855/1724] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:39 2022] POST /identity/v3/users => generated 288 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:40.140677 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e1e80-8697-4e4f-80bc-bf091a67123b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:40.168187 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98e1e80-8697-4e4f-80bc-bf091a67123b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:40.208421 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1cb5d-3091-45a9-a30a-6ef08d1a2f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:40.209482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1cb5d-3091-45a9-a30a-6ef08d1a2f3d admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:40.210347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1cb5d-3091-45a9-a30a-6ef08d1a2f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:40.210634 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1cb5d-3091-45a9-a30a-6ef08d1a2f3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:40.211351 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1cb5d-3091-45a9-a30a-6ef08d1a2f3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:40.223675 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1cb5d-3091-45a9-a30a-6ef08d1a2f3d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:40.266161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15742a1b-aa8b-4b3b-927f-8375c8d1b2ea admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:40.276040 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 870/1725] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:40.300391 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15742a1b-aa8b-4b3b-927f-8375c8d1b2ea admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:40.311503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d0961-2417-44ec-8257-2302c260fb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:40.312428 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d0961-2417-44ec-8257-2302c260fb7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:40.313265 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d0961-2417-44ec-8257-2302c260fb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:40.313524 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d0961-2417-44ec-8257-2302c260fb7b admin admin] PATH_INFO: `/v3/projects/a8b1fcc1df0b47d8937792e9d2d3b87c/users/7797567d4e404cc5885821ae1f195e58/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:40.364505 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d0961-2417-44ec-8257-2302c260fb7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8b1fcc1df0b47d8937792e9d2d3b87c, user_id=7797567d4e404cc5885821ae1f195e58, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:40.380126 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d0961-2417-44ec-8257-2302c260fb7b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:40.407914 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 856/1726] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:40 2022] PUT /identity/v3/projects/a8b1fcc1df0b47d8937792e9d2d3b87c/users/7797567d4e404cc5885821ae1f195e58/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:40.411525 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1cb5d-3091-45a9-a30a-6ef08d1a2f3d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:40.432623 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1cb5d-3091-45a9-a30a-6ef08d1a2f3d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:40.441204 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756d1c5-b2f4-4f32-b0c3-63d4b8627102 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:40.441502 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756d1c5-b2f4-4f32-b0c3-63d4b8627102 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:40.442976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756d1c5-b2f4-4f32-b0c3-63d4b8627102 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:40.444699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 857/1727] 10.222.0.20 () {54 vars in 921 bytes} [Sat Aug 6 12:30:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:30:40.505565 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f1100-e0ad-4a70-b355-8f3315cc3831 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:40.505852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f1100-e0ad-4a70-b355-8f3315cc3831 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:40.506121 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f1100-e0ad-4a70-b355-8f3315cc3831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:40.530213 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:40.546962 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c71f1100-e0ad-4a70-b355-8f3315cc3831 None None] MFA Rules not processed for user `524b728bf63c4ca88e0f25e531cf5056`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:40.552105 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c71f1100-e0ad-4a70-b355-8f3315cc3831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:40.632416 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:40.654714 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f089b-4199-4018-8f6c-a38e82f13cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:40.656159 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f089b-4199-4018-8f6c-a38e82f13cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:40.656686 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f089b-4199-4018-8f6c-a38e82f13cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:40.657123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f089b-4199-4018-8f6c-a38e82f13cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:40.657956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f089b-4199-4018-8f6c-a38e82f13cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:40.659682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f089b-4199-4018-8f6c-a38e82f13cb5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:40.680775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 871/1728] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:40.682393 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27115f-da28-4828-9f71-17399e996679 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:40.682885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27115f-da28-4828-9f71-17399e996679 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:40.688783 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27115f-da28-4828-9f71-17399e996679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:40.737374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-df27115f-da28-4828-9f71-17399e996679 None None] MFA Rules not processed for user `524b728bf63c4ca88e0f25e531cf5056`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:40.739365 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-df27115f-da28-4828-9f71-17399e996679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:40.845353 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 858/1729] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Aug 6 12:30:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:40.849417 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f1100-e0ad-4a70-b355-8f3315cc3831 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:40.854113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c71f1100-e0ad-4a70-b355-8f3315cc3831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:40.907278 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f1100-e0ad-4a70-b355-8f3315cc3831 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:40.917087 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e2843-3ff1-4f75-bb37-755ca6e148ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:40.917749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e2843-3ff1-4f75-bb37-755ca6e148ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:40.918035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e2843-3ff1-4f75-bb37-755ca6e148ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:40.918317 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e2843-3ff1-4f75-bb37-755ca6e148ee admin admin] PATH_INFO: `/v3/projects/abd205f7acc3405e872cca67f4549aae` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:40.921021 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e2843-3ff1-4f75-bb37-755ca6e148ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=abd205f7acc3405e872cca67f4549aae)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:40.922950 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e2843-3ff1-4f75-bb37-755ca6e148ee admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:41.031605 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 872/1730] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Aug 6 12:30:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:41.033536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df27115f-da28-4828-9f71-17399e996679 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:41.054770 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-dc9e2843-3ff1-4f75-bb37-755ca6e148ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abd205f7acc3405e872cca67f4549aae', 'request_id': 'req-dc9e2843-3ff1-4f75-bb37-755ca6e148ee'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:41.076717 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-dc9e2843-3ff1-4f75-bb37-755ca6e148ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abd205f7acc3405e872cca67f4549aae', 'request_id': 'req-dc9e2843-3ff1-4f75-bb37-755ca6e148ee'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:41.077856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df27115f-da28-4828-9f71-17399e996679 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:41.100985 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-dc9e2843-3ff1-4f75-bb37-755ca6e148ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abd205f7acc3405e872cca67f4549aae', 'request_id': 'req-dc9e2843-3ff1-4f75-bb37-755ca6e148ee'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:41.142652 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-dc9e2843-3ff1-4f75-bb37-755ca6e148ee admin admin] The token cache is being invalidate because project abd205f7acc3405e872cca67f4549aae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:30:41.142957 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-dc9e2843-3ff1-4f75-bb37-755ca6e148ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:41.150131 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-dc9e2843-3ff1-4f75-bb37-755ca6e148ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:41.152222 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 859/1731] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:30:40 2022] DELETE /identity/v3/projects/abd205f7acc3405e872cca67f4549aae => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:41.156758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e2843-3ff1-4f75-bb37-755ca6e148ee admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:41.158484 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-dc9e2843-3ff1-4f75-bb37-755ca6e148ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:41.252129 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a15e7-6cb1-42ff-b63f-aebc3d33a232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:41.253346 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a15e7-6cb1-42ff-b63f-aebc3d33a232 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:41.254886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a15e7-6cb1-42ff-b63f-aebc3d33a232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:41.256070 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a15e7-6cb1-42ff-b63f-aebc3d33a232 admin admin] PATH_INFO: `/v3/projects/a8b1fcc1df0b47d8937792e9d2d3b87c/users/7797567d4e404cc5885821ae1f195e58/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:41.260107 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79a15e7-6cb1-42ff-b63f-aebc3d33a232 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8b1fcc1df0b47d8937792e9d2d3b87c, user_id=7797567d4e404cc5885821ae1f195e58, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:41.271664 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79a15e7-6cb1-42ff-b63f-aebc3d33a232 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:41.274791 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e2843-3ff1-4f75-bb37-755ca6e148ee admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:41.302360 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d0d3a-c758-402a-ace7-2e1a3a5cea89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:41.303518 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d0d3a-c758-402a-ace7-2e1a3a5cea89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:41.303940 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d0d3a-c758-402a-ace7-2e1a3a5cea89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:41.304589 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d0d3a-c758-402a-ace7-2e1a3a5cea89 admin admin] PATH_INFO: `/v3/users/e107c439e2c8451082ee1e7a8d418fe5` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:41.305307 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 873/1732] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:41 2022] PUT /identity/v3/projects/a8b1fcc1df0b47d8937792e9d2d3b87c/users/7797567d4e404cc5885821ae1f195e58/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:41.306452 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d0d3a-c758-402a-ace7-2e1a3a5cea89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e107c439e2c8451082ee1e7a8d418fe5)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:41.312803 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d0d3a-c758-402a-ace7-2e1a3a5cea89 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:41.315730 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99d27d-ea99-4a56-9bfb-75f5dd641a92 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:41.316021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99d27d-ea99-4a56-9bfb-75f5dd641a92 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:41.316434 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99d27d-ea99-4a56-9bfb-75f5dd641a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:41.356924 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-cb99d27d-ea99-4a56-9bfb-75f5dd641a92 None None] MFA Rules not processed for user `7797567d4e404cc5885821ae1f195e58`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:41.358173 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cb99d27d-ea99-4a56-9bfb-75f5dd641a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:41.367661 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:41.412325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-714d0d3a-c758-402a-ace7-2e1a3a5cea89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e107c439e2c8451082ee1e7a8d418fe5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:41.427181 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-714d0d3a-c758-402a-ace7-2e1a3a5cea89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e107c439e2c8451082ee1e7a8d418fe5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:41.469689 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 874/1733] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:41 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:30:41.481080 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-714d0d3a-c758-402a-ace7-2e1a3a5cea89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e107c439e2c8451082ee1e7a8d418fe5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:41.483322 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-714d0d3a-c758-402a-ace7-2e1a3a5cea89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e107c439e2c8451082ee1e7a8d418fe5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:41.484071 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-714d0d3a-c758-402a-ace7-2e1a3a5cea89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e107c439e2c8451082ee1e7a8d418fe5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:41.484292 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99d27d-ea99-4a56-9bfb-75f5dd641a92 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:41.486387 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cb99d27d-ea99-4a56-9bfb-75f5dd641a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:41.500408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:41.532762 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 860/1734] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:30:41 2022] DELETE /identity/v3/users/e107c439e2c8451082ee1e7a8d418fe5 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:41.644765 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99d27d-ea99-4a56-9bfb-75f5dd641a92 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:41.767037 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678628f-7a90-46f8-8b95-190e70d31460 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:41.767682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678628f-7a90-46f8-8b95-190e70d31460 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:41.767970 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678628f-7a90-46f8-8b95-190e70d31460 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:41.768257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678628f-7a90-46f8-8b95-190e70d31460 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:41.781174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4678628f-7a90-46f8-8b95-190e70d31460 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:41.813911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d0d3a-c758-402a-ace7-2e1a3a5cea89 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:41.815346 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-714d0d3a-c758-402a-ace7-2e1a3a5cea89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:42.013981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d0d3a-c758-402a-ace7-2e1a3a5cea89 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:42.025261 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee8df6-bdc7-4442-bfc0-3a75a371237f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:42.026276 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee8df6-bdc7-4442-bfc0-3a75a371237f service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:42.026532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee8df6-bdc7-4442-bfc0-3a75a371237f service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:42.026893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee8df6-bdc7-4442-bfc0-3a75a371237f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:42.029618 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-61ee8df6-bdc7-4442-bfc0-3a75a371237f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:42.059634 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678628f-7a90-46f8-8b95-190e70d31460 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:42.061555 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678628f-7a90-46f8-8b95-190e70d31460 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:42.078142 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ee8df6-bdc7-4442-bfc0-3a75a371237f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:42.080732 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ee8df6-bdc7-4442-bfc0-3a75a371237f service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:42.088500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 875/1735] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:41 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:42.109583 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 861/1736] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:42.329457 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678628f-7a90-46f8-8b95-190e70d31460 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:42.330959 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4678628f-7a90-46f8-8b95-190e70d31460 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:42.379646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678628f-7a90-46f8-8b95-190e70d31460 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:42.409436 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cab0ab6-d5a4-451f-a471-b53bca12a658 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:42.409919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab0ab6-d5a4-451f-a471-b53bca12a658 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:42.410261 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab0ab6-d5a4-451f-a471-b53bca12a658 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:42.410395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab0ab6-d5a4-451f-a471-b53bca12a658 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:42.413271 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6cab0ab6-d5a4-451f-a471-b53bca12a658 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:42.448473 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cab0ab6-d5a4-451f-a471-b53bca12a658 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:42.462828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cab0ab6-d5a4-451f-a471-b53bca12a658 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:42.535416 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 876/1737] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:42.793404 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee8df6-bdc7-4442-bfc0-3a75a371237f service nova] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:42.797452 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-61ee8df6-bdc7-4442-bfc0-3a75a371237f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:42.932886 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee8df6-bdc7-4442-bfc0-3a75a371237f service nova] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:42.940125 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11307414-de55-4d19-87d1-ac509f6a5170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:42.941300 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11307414-de55-4d19-87d1-ac509f6a5170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:42.942258 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11307414-de55-4d19-87d1-ac509f6a5170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:42.942679 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11307414-de55-4d19-87d1-ac509f6a5170 admin admin] PATH_INFO: `/v3/projects/49da3e9e24d64d01b7d343a562830e31` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:42.944788 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11307414-de55-4d19-87d1-ac509f6a5170 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49da3e9e24d64d01b7d343a562830e31)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:42.947773 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11307414-de55-4d19-87d1-ac509f6a5170 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:42.980264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cab0ab6-d5a4-451f-a471-b53bca12a658 service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:42.981865 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6cab0ab6-d5a4-451f-a471-b53bca12a658 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:43.013621 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:43.016742 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cab0ab6-d5a4-451f-a471-b53bca12a658 service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:43.033062 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b31284-3f3d-4404-8aa2-516d6d941969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:43.035208 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b31284-3f3d-4404-8aa2-516d6d941969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:43.035529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b31284-3f3d-4404-8aa2-516d6d941969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:43.036979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b31284-3f3d-4404-8aa2-516d6d941969 admin admin] PATH_INFO: `/v3/users/5c14ab0d76364390b1690cfcee968791` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:43.039929 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b31284-3f3d-4404-8aa2-516d6d941969 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c14ab0d76364390b1690cfcee968791)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:43.042544 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b31284-3f3d-4404-8aa2-516d6d941969 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:43.068688 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-11307414-de55-4d19-87d1-ac509f6a5170 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49da3e9e24d64d01b7d343a562830e31', 'request_id': 'req-11307414-de55-4d19-87d1-ac509f6a5170'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:43.076788 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-11307414-de55-4d19-87d1-ac509f6a5170 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49da3e9e24d64d01b7d343a562830e31', 'request_id': 'req-11307414-de55-4d19-87d1-ac509f6a5170'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:43.107449 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-11307414-de55-4d19-87d1-ac509f6a5170 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49da3e9e24d64d01b7d343a562830e31', 'request_id': 'req-11307414-de55-4d19-87d1-ac509f6a5170'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:43.132598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-85b31284-3f3d-4404-8aa2-516d6d941969 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c14ab0d76364390b1690cfcee968791'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:43.133756 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-11307414-de55-4d19-87d1-ac509f6a5170 admin admin] The token cache is being invalidate because project 49da3e9e24d64d01b7d343a562830e31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:30:43.134262 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-11307414-de55-4d19-87d1-ac509f6a5170 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:43.135414 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-11307414-de55-4d19-87d1-ac509f6a5170 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:43.142302 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 862/1738] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:30:42 2022] DELETE /identity/v3/projects/49da3e9e24d64d01b7d343a562830e31 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:43.142302 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11307414-de55-4d19-87d1-ac509f6a5170 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:43.145077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:43.147783 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-11307414-de55-4d19-87d1-ac509f6a5170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:43.156499 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-85b31284-3f3d-4404-8aa2-516d6d941969 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c14ab0d76364390b1690cfcee968791'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:43.157707 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-85b31284-3f3d-4404-8aa2-516d6d941969 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c14ab0d76364390b1690cfcee968791'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:43.158788 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-85b31284-3f3d-4404-8aa2-516d6d941969 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c14ab0d76364390b1690cfcee968791'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:43.173748 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-85b31284-3f3d-4404-8aa2-516d6d941969 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c14ab0d76364390b1690cfcee968791'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:43.199651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 877/1739] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:30:42 2022] DELETE /identity/v3/users/5c14ab0d76364390b1690cfcee968791 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:43.199651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fef94d-a724-4925-8fa0-8f33ffa791a6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:43.199651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fef94d-a724-4925-8fa0-8f33ffa791a6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:43.199651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fef94d-a724-4925-8fa0-8f33ffa791a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:43.224698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-98fef94d-a724-4925-8fa0-8f33ffa791a6 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:43.225985 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-98fef94d-a724-4925-8fa0-8f33ffa791a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:43.234028 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11307414-de55-4d19-87d1-ac509f6a5170 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:43.295882 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 878/1740] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:43.295882 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88f961-ecf8-44fd-8049-9cdb8709fc2a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:43.295882 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88f961-ecf8-44fd-8049-9cdb8709fc2a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:43.295882 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88f961-ecf8-44fd-8049-9cdb8709fc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:43.298159 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e007a0-106a-4520-8797-bca6e9f5a162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:43.298812 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e007a0-106a-4520-8797-bca6e9f5a162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:43.299095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e007a0-106a-4520-8797-bca6e9f5a162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:43.299370 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e007a0-106a-4520-8797-bca6e9f5a162 admin admin] PATH_INFO: `/v3/projects/a5308b4ab1c94ae5b1735813b7d215aa` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:43.300570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e007a0-106a-4520-8797-bca6e9f5a162 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5308b4ab1c94ae5b1735813b7d215aa)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:43.303022 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e007a0-106a-4520-8797-bca6e9f5a162 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:43.314262 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5c88f961-ecf8-44fd-8049-9cdb8709fc2a None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:43.315750 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5c88f961-ecf8-44fd-8049-9cdb8709fc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:43.325884 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 879/1741] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:43.326903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c88f961-ecf8-44fd-8049-9cdb8709fc2a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:43.333132 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5c88f961-ecf8-44fd-8049-9cdb8709fc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:43.347647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c88f961-ecf8-44fd-8049-9cdb8709fc2a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:43.365756 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-24e007a0-106a-4520-8797-bca6e9f5a162 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5308b4ab1c94ae5b1735813b7d215aa', 'request_id': 'req-24e007a0-106a-4520-8797-bca6e9f5a162'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:43.373214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-24e007a0-106a-4520-8797-bca6e9f5a162 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5308b4ab1c94ae5b1735813b7d215aa', 'request_id': 'req-24e007a0-106a-4520-8797-bca6e9f5a162'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:43.383389 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-24e007a0-106a-4520-8797-bca6e9f5a162 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5308b4ab1c94ae5b1735813b7d215aa', 'request_id': 'req-24e007a0-106a-4520-8797-bca6e9f5a162'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:43.384960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:43.414013 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-24e007a0-106a-4520-8797-bca6e9f5a162 admin admin] The token cache is being invalidate because project a5308b4ab1c94ae5b1735813b7d215aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:30:43.414219 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-24e007a0-106a-4520-8797-bca6e9f5a162 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:43.416415 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-24e007a0-106a-4520-8797-bca6e9f5a162 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:43.417695 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 863/1742] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:30:43 2022] DELETE /identity/v3/projects/a5308b4ab1c94ae5b1735813b7d215aa => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:43.418929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b486596-28da-4d99-8aab-8b548dba1fa0 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:43.419215 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b486596-28da-4d99-8aab-8b548dba1fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:43.419576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b486596-28da-4d99-8aab-8b548dba1fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:43.423756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce760fdb-81a0-4f63-b2fd-f671c14aa1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:43.424438 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce760fdb-81a0-4f63-b2fd-f671c14aa1d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:43.424724 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce760fdb-81a0-4f63-b2fd-f671c14aa1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:43.424992 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce760fdb-81a0-4f63-b2fd-f671c14aa1d1 admin admin] PATH_INFO: `/v3/users/1a19e414e5ab48fb8fd2615e694dec41` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:43.428329 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce760fdb-81a0-4f63-b2fd-f671c14aa1d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a19e414e5ab48fb8fd2615e694dec41)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:43.428638 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce760fdb-81a0-4f63-b2fd-f671c14aa1d1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:43.448143 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-9b486596-28da-4d99-8aab-8b548dba1fa0 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:43.449400 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9b486596-28da-4d99-8aab-8b548dba1fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:43.461536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 864/1743] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:43.462493 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b486596-28da-4d99-8aab-8b548dba1fa0 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:43.463943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9b486596-28da-4d99-8aab-8b548dba1fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:43.494147 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ce760fdb-81a0-4f63-b2fd-f671c14aa1d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a19e414e5ab48fb8fd2615e694dec41'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:43.505866 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:43.507059 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ce760fdb-81a0-4f63-b2fd-f671c14aa1d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a19e414e5ab48fb8fd2615e694dec41'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:43.507831 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ce760fdb-81a0-4f63-b2fd-f671c14aa1d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a19e414e5ab48fb8fd2615e694dec41'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:43.508834 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ce760fdb-81a0-4f63-b2fd-f671c14aa1d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a19e414e5ab48fb8fd2615e694dec41'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:43.516931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ce760fdb-81a0-4f63-b2fd-f671c14aa1d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a19e414e5ab48fb8fd2615e694dec41'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:43.522707 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b486596-28da-4d99-8aab-8b548dba1fa0 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:43.526471 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05599440-d74c-4029-a513-32320f99d176 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:43.527106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05599440-d74c-4029-a513-32320f99d176 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:43.527376 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05599440-d74c-4029-a513-32320f99d176 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:43.527649 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05599440-d74c-4029-a513-32320f99d176 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:43.529457 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-05599440-d74c-4029-a513-32320f99d176 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:43.529806 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 880/1744] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:30:43 2022] DELETE /identity/v3/users/1a19e414e5ab48fb8fd2615e694dec41 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:43.531039 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d66639-5d67-4014-88f4-2926e32d4c70 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:43.531326 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d66639-5d67-4014-88f4-2926e32d4c70 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:43.531601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d66639-5d67-4014-88f4-2926e32d4c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:43.543596 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05599440-d74c-4029-a513-32320f99d176 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:43.545280 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05599440-d74c-4029-a513-32320f99d176 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:43.552389 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-99d66639-5d67-4014-88f4-2926e32d4c70 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:43.553588 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-99d66639-5d67-4014-88f4-2926e32d4c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:43.556792 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 865/1745] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:43.557758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05599440-d74c-4029-a513-32320f99d176 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:43.559040 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-05599440-d74c-4029-a513-32320f99d176 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:43.571096 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05599440-d74c-4029-a513-32320f99d176 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:43.620932 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 881/1746] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:43.628242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6b42d-fa80-486f-bfee-20cae40e8899 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:43.628242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6b42d-fa80-486f-bfee-20cae40e8899 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:43.628242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6b42d-fa80-486f-bfee-20cae40e8899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:43.646341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f5e6b42d-fa80-486f-bfee-20cae40e8899 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:43.647654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f5e6b42d-fa80-486f-bfee-20cae40e8899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:43.647787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad176d5b-65ce-41ed-973e-6f26261e175b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:43.648920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad176d5b-65ce-41ed-973e-6f26261e175b admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:43.649210 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad176d5b-65ce-41ed-973e-6f26261e175b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:43.649489 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad176d5b-65ce-41ed-973e-6f26261e175b admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:43.650190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad176d5b-65ce-41ed-973e-6f26261e175b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:30:43.650496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad176d5b-65ce-41ed-973e-6f26261e175b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:43.652150 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad176d5b-65ce-41ed-973e-6f26261e175b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:43.656325 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 882/1747] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:43.657186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47998086-82fa-4ff1-95e8-fc6fb2a6c7f9 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:43.657507 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47998086-82fa-4ff1-95e8-fc6fb2a6c7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:43.657739 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47998086-82fa-4ff1-95e8-fc6fb2a6c7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:43.673955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 866/1748] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:30:43.673955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad176d5b-65ce-41ed-973e-6f26261e175b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:43.677319 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ad176d5b-65ce-41ed-973e-6f26261e175b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:43.680905 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-47998086-82fa-4ff1-95e8-fc6fb2a6c7f9 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:43.682084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-47998086-82fa-4ff1-95e8-fc6fb2a6c7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:43.692420 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 883/1749] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:43.693197 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47998086-82fa-4ff1-95e8-fc6fb2a6c7f9 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:43.703377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47998086-82fa-4ff1-95e8-fc6fb2a6c7f9 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:43.709613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b8364-6abc-4501-9dad-4faddb9b6e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:43.710239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b8364-6abc-4501-9dad-4faddb9b6e77 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:43.710527 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b8364-6abc-4501-9dad-4faddb9b6e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:43.710852 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b8364-6abc-4501-9dad-4faddb9b6e77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:43.711465 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b8364-6abc-4501-9dad-4faddb9b6e77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:43.713424 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b8364-6abc-4501-9dad-4faddb9b6e77 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:43.753649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 884/1750] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:43 2022] POST /identity/v3/projects => generated 359 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:43.754293 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad176d5b-65ce-41ed-973e-6f26261e175b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:43.754610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b8364-6abc-4501-9dad-4faddb9b6e77 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:43.803887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:43.816894 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc07620-0e63-4ff5-80df-bc405b5c48f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:43.817514 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc07620-0e63-4ff5-80df-bc405b5c48f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:43.817818 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc07620-0e63-4ff5-80df-bc405b5c48f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:43.818085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc07620-0e63-4ff5-80df-bc405b5c48f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:43.830983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b8364-6abc-4501-9dad-4faddb9b6e77 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:43.836363 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9e64b-737a-4959-9076-60b55d5b205f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:43.836991 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9e64b-737a-4959-9076-60b55d5b205f admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:43.837274 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9e64b-737a-4959-9076-60b55d5b205f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:43.837548 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9e64b-737a-4959-9076-60b55d5b205f admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:43.838249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9e64b-737a-4959-9076-60b55d5b205f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:30:43.838550 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9e64b-737a-4959-9076-60b55d5b205f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:43.842892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9e64b-737a-4959-9076-60b55d5b205f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:43.858078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 885/1751] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:30:43.859462 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9e64b-737a-4959-9076-60b55d5b205f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:43.871225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9e64b-737a-4959-9076-60b55d5b205f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:43.876350 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b0e7f-b353-4072-8643-9ee00817f349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:43.877039 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b0e7f-b353-4072-8643-9ee00817f349 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:43.877415 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b0e7f-b353-4072-8643-9ee00817f349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:43.877758 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b0e7f-b353-4072-8643-9ee00817f349 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:43.878494 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b0e7f-b353-4072-8643-9ee00817f349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:43.880579 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b0e7f-b353-4072-8643-9ee00817f349 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:43.885907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc07620-0e63-4ff5-80df-bc405b5c48f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:43.887364 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc07620-0e63-4ff5-80df-bc405b5c48f2 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:43.897987 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 867/1752] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:43.898656 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc07620-0e63-4ff5-80df-bc405b5c48f2 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:43.904571 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 886/1753] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:43 2022] POST /identity/v3/users => generated 280 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:43.908216 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc07620-0e63-4ff5-80df-bc405b5c48f2 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:43.910959 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b0e7f-b353-4072-8643-9ee00817f349 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:43.913409 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-608b0e7f-b353-4072-8643-9ee00817f349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:43.916582 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22296c3d-3e26-4663-b3ae-fc9863527790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:43.917351 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22296c3d-3e26-4663-b3ae-fc9863527790 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:43.917687 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22296c3d-3e26-4663-b3ae-fc9863527790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:43.918120 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22296c3d-3e26-4663-b3ae-fc9863527790 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:43.919422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22296c3d-3e26-4663-b3ae-fc9863527790 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:43.921819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22296c3d-3e26-4663-b3ae-fc9863527790 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:43.926090 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b0e7f-b353-4072-8643-9ee00817f349 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:43.932021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85dd38-1d45-46f6-866a-479e40d4d343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:43.933306 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85dd38-1d45-46f6-866a-479e40d4d343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:43.933598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85dd38-1d45-46f6-866a-479e40d4d343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:43.933904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85dd38-1d45-46f6-866a-479e40d4d343 admin admin] PATH_INFO: `/v3/projects/dc9c1a0a10e74d6eb0bc048c0e032b73` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:43.935473 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85dd38-1d45-46f6-866a-479e40d4d343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc9c1a0a10e74d6eb0bc048c0e032b73)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:43.936841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: WARNING py.warnings [None req-fe85dd38-1d45-46f6-866a-479e40d4d343 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 06 12:30:43.936841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: warnings.warn(msg) Aug 06 12:30:43.936841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: Aug 06 12:30:43.938069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85dd38-1d45-46f6-866a-479e40d4d343 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:43.956374 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 868/1754] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:43 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:43.956374 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22296c3d-3e26-4663-b3ae-fc9863527790 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:43.966676 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:44.021891 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-fe85dd38-1d45-46f6-866a-479e40d4d343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc9c1a0a10e74d6eb0bc048c0e032b73', 'request_id': 'req-fe85dd38-1d45-46f6-866a-479e40d4d343'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:44.031173 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-fe85dd38-1d45-46f6-866a-479e40d4d343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc9c1a0a10e74d6eb0bc048c0e032b73', 'request_id': 'req-fe85dd38-1d45-46f6-866a-479e40d4d343'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:44.042064 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22296c3d-3e26-4663-b3ae-fc9863527790 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:44.049995 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-fe85dd38-1d45-46f6-866a-479e40d4d343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc9c1a0a10e74d6eb0bc048c0e032b73', 'request_id': 'req-fe85dd38-1d45-46f6-866a-479e40d4d343'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:44.079066 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-fe85dd38-1d45-46f6-866a-479e40d4d343 admin admin] The token cache is being invalidate because project dc9c1a0a10e74d6eb0bc048c0e032b73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:30:44.079382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-fe85dd38-1d45-46f6-866a-479e40d4d343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:44.080108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-fe85dd38-1d45-46f6-866a-479e40d4d343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:44.081740 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 887/1755] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:30:43 2022] DELETE /identity/v3/projects/dc9c1a0a10e74d6eb0bc048c0e032b73 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:44.083144 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85dd38-1d45-46f6-866a-479e40d4d343 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:44.084543 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fe85dd38-1d45-46f6-866a-479e40d4d343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:44.163895 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af53b23-a795-4a42-8612-dc2e99c4b3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:44.164547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af53b23-a795-4a42-8612-dc2e99c4b3e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:44.164834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af53b23-a795-4a42-8612-dc2e99c4b3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:44.165107 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af53b23-a795-4a42-8612-dc2e99c4b3e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:44.166160 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af53b23-a795-4a42-8612-dc2e99c4b3e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:44.167563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af53b23-a795-4a42-8612-dc2e99c4b3e7 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:44.177029 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85dd38-1d45-46f6-866a-479e40d4d343 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:44.183296 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df53e3-af0c-4153-896b-8cae35043270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:44.184246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df53e3-af0c-4153-896b-8cae35043270 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:44.184773 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df53e3-af0c-4153-896b-8cae35043270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:44.185501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df53e3-af0c-4153-896b-8cae35043270 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:44.189326 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df53e3-af0c-4153-896b-8cae35043270 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:44.189494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 869/1756] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:44.189494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af53b23-a795-4a42-8612-dc2e99c4b3e7 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:44.191852 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df53e3-af0c-4153-896b-8cae35043270 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:44.192707 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9af53b23-a795-4a42-8612-dc2e99c4b3e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:44.213673 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af53b23-a795-4a42-8612-dc2e99c4b3e7 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:44.219261 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114fafd-1051-45b5-9cf5-709fd74dda08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:44.220040 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114fafd-1051-45b5-9cf5-709fd74dda08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:44.220558 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114fafd-1051-45b5-9cf5-709fd74dda08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:44.220864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114fafd-1051-45b5-9cf5-709fd74dda08 admin admin] PATH_INFO: `/v3/users/7797567d4e404cc5885821ae1f195e58` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:44.222380 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114fafd-1051-45b5-9cf5-709fd74dda08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7797567d4e404cc5885821ae1f195e58)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:44.225054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114fafd-1051-45b5-9cf5-709fd74dda08 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:44.231875 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:44.233158 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 888/1757] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:44 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:44.234455 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df53e3-af0c-4153-896b-8cae35043270 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:44.237082 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-58df53e3-af0c-4153-896b-8cae35043270 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:44.255473 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df53e3-af0c-4153-896b-8cae35043270 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:44.262282 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8df20e-7bad-470a-b647-e235649a82d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:44.263881 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8df20e-7bad-470a-b647-e235649a82d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:44.264483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8df20e-7bad-470a-b647-e235649a82d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:44.264959 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8df20e-7bad-470a-b647-e235649a82d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:44.266004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8df20e-7bad-470a-b647-e235649a82d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:44.268157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8df20e-7bad-470a-b647-e235649a82d3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:44.285584 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 889/1758] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:44.286793 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8df20e-7bad-470a-b647-e235649a82d3 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:44.290583 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4114fafd-1051-45b5-9cf5-709fd74dda08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7797567d4e404cc5885821ae1f195e58'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:44.299114 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8df20e-7bad-470a-b647-e235649a82d3 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:44.301944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4114fafd-1051-45b5-9cf5-709fd74dda08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7797567d4e404cc5885821ae1f195e58'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:44.305222 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c3c7e-cb46-42ba-80b8-5694ffcd0c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:44.306316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c3c7e-cb46-42ba-80b8-5694ffcd0c41 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:44.306664 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c3c7e-cb46-42ba-80b8-5694ffcd0c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:44.307003 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c3c7e-cb46-42ba-80b8-5694ffcd0c41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:44.307790 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c3c7e-cb46-42ba-80b8-5694ffcd0c41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:44.310082 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c3c7e-cb46-42ba-80b8-5694ffcd0c41 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:44.319855 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4114fafd-1051-45b5-9cf5-709fd74dda08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7797567d4e404cc5885821ae1f195e58'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:44.320650 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4114fafd-1051-45b5-9cf5-709fd74dda08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7797567d4e404cc5885821ae1f195e58'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:44.321281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4114fafd-1051-45b5-9cf5-709fd74dda08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7797567d4e404cc5885821ae1f195e58'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:44.326151 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 890/1759] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:44.326151 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c3c7e-cb46-42ba-80b8-5694ffcd0c41 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:44.326773 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1f1c3c7e-cb46-42ba-80b8-5694ffcd0c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:44.340586 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 870/1760] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:30:44 2022] DELETE /identity/v3/users/7797567d4e404cc5885821ae1f195e58 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:44.341709 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114fafd-1051-45b5-9cf5-709fd74dda08 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:44.344950 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c3c7e-cb46-42ba-80b8-5694ffcd0c41 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:44.346364 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4114fafd-1051-45b5-9cf5-709fd74dda08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:44.429407 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad471ec-fa0d-4c19-82fb-97232eb3cff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:44.430231 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad471ec-fa0d-4c19-82fb-97232eb3cff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:44.430576 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad471ec-fa0d-4c19-82fb-97232eb3cff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:44.430867 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad471ec-fa0d-4c19-82fb-97232eb3cff4 admin admin] PATH_INFO: `/v3/projects/a30bc03cd15949cca57a8a810d820747/users/6db1e2303fc64d8ea7e479dab966c62d/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:44.452952 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad471ec-fa0d-4c19-82fb-97232eb3cff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a30bc03cd15949cca57a8a810d820747, user_id=6db1e2303fc64d8ea7e479dab966c62d, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:44.453766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114fafd-1051-45b5-9cf5-709fd74dda08 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:44.455865 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad471ec-fa0d-4c19-82fb-97232eb3cff4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:44.460022 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:44.460775 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a367d-e672-41a1-b12e-1c68d2cfd93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:44.461433 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a367d-e672-41a1-b12e-1c68d2cfd93f admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:44.461927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a367d-e672-41a1-b12e-1c68d2cfd93f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:44.462400 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a367d-e672-41a1-b12e-1c68d2cfd93f admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:44.463592 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a367d-e672-41a1-b12e-1c68d2cfd93f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:44.465775 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a367d-e672-41a1-b12e-1c68d2cfd93f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:44.483294 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 871/1761] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:44.485268 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a367d-e672-41a1-b12e-1c68d2cfd93f admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:44.485620 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 891/1762] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:44 2022] PUT /identity/v3/projects/a30bc03cd15949cca57a8a810d820747/users/6db1e2303fc64d8ea7e479dab966c62d/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:44.498570 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833b58b-7909-40ad-95b0-c3e2b64fc079 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:44.500157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833b58b-7909-40ad-95b0-c3e2b64fc079 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:44.503949 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833b58b-7909-40ad-95b0-c3e2b64fc079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:44.512272 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a367d-e672-41a1-b12e-1c68d2cfd93f admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:44.539495 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b833b58b-7909-40ad-95b0-c3e2b64fc079 None None] MFA Rules not processed for user `6db1e2303fc64d8ea7e479dab966c62d`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:44.541193 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b833b58b-7909-40ad-95b0-c3e2b64fc079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:44.620202 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d25e4-e3ff-4f36-9732-a552b5c25b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:44.620932 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d25e4-e3ff-4f36-9732-a552b5c25b41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:44.623755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d25e4-e3ff-4f36-9732-a552b5c25b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:44.623913 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d25e4-e3ff-4f36-9732-a552b5c25b41 admin admin] PATH_INFO: `/v3/projects/712607f8374e4c6fa783c34f0d59a45b/users/5cd8417063fd49cbb5488c16501853ca/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:44.635863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 892/1763] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:44 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:44.641110 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3d25e4-e3ff-4f36-9732-a552b5c25b41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=712607f8374e4c6fa783c34f0d59a45b, user_id=5cd8417063fd49cbb5488c16501853ca, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:44.649904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3d25e4-e3ff-4f36-9732-a552b5c25b41 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:44.681172 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833b58b-7909-40ad-95b0-c3e2b64fc079 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:44.682622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b833b58b-7909-40ad-95b0-c3e2b64fc079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:44.698733 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 872/1764] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:44 2022] PUT /identity/v3/projects/712607f8374e4c6fa783c34f0d59a45b/users/5cd8417063fd49cbb5488c16501853ca/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:44.700883 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734657c-0fbf-4d92-a2d9-298d42f319ad None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:44.701174 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734657c-0fbf-4d92-a2d9-298d42f319ad None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:44.701456 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734657c-0fbf-4d92-a2d9-298d42f319ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:44.724852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:44.781568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-d734657c-0fbf-4d92-a2d9-298d42f319ad None None] MFA Rules not processed for user `5cd8417063fd49cbb5488c16501853ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:44.782832 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d734657c-0fbf-4d92-a2d9-298d42f319ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:44.830321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833b58b-7909-40ad-95b0-c3e2b64fc079 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:44.893223 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 873/1765] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:30:44.893223 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734657c-0fbf-4d92-a2d9-298d42f319ad None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:44.893223 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d734657c-0fbf-4d92-a2d9-298d42f319ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:44.934705 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e52c6-d447-494b-ba9d-26c73330db12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:44.935282 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e52c6-d447-494b-ba9d-26c73330db12 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:44.935561 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e52c6-d447-494b-ba9d-26c73330db12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:44.935835 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e52c6-d447-494b-ba9d-26c73330db12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:44.969687 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734657c-0fbf-4d92-a2d9-298d42f319ad None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:44.975293 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2ebb3-f894-4822-a7dd-a0179a8f2046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:44.975953 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2ebb3-f894-4822-a7dd-a0179a8f2046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:44.976234 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2ebb3-f894-4822-a7dd-a0179a8f2046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:44.976672 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2ebb3-f894-4822-a7dd-a0179a8f2046 admin admin] PATH_INFO: `/v3/projects/386778dcad4d45c7bc5c343b735983b2` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:44.977908 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c2ebb3-f894-4822-a7dd-a0179a8f2046 admin admin] RBAC: Authorizing `identity:delete_project(project_id=386778dcad4d45c7bc5c343b735983b2)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:44.980084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c2ebb3-f894-4822-a7dd-a0179a8f2046 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:45.037555 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:45.038647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-52c2ebb3-f894-4822-a7dd-a0179a8f2046 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '386778dcad4d45c7bc5c343b735983b2', 'request_id': 'req-52c2ebb3-f894-4822-a7dd-a0179a8f2046'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:45.044369 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e52c6-d447-494b-ba9d-26c73330db12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:45.045840 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-52c2ebb3-f894-4822-a7dd-a0179a8f2046 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '386778dcad4d45c7bc5c343b735983b2', 'request_id': 'req-52c2ebb3-f894-4822-a7dd-a0179a8f2046'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:45.046324 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e52c6-d447-494b-ba9d-26c73330db12 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:45.057500 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-52c2ebb3-f894-4822-a7dd-a0179a8f2046 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '386778dcad4d45c7bc5c343b735983b2', 'request_id': 'req-52c2ebb3-f894-4822-a7dd-a0179a8f2046'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:45.062687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 893/1766] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:44 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:45.063607 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e52c6-d447-494b-ba9d-26c73330db12 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:45.077435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e52c6-d447-494b-ba9d-26c73330db12 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:45.083711 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-52c2ebb3-f894-4822-a7dd-a0179a8f2046 admin admin] The token cache is being invalidate because project 386778dcad4d45c7bc5c343b735983b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:30:45.084378 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-52c2ebb3-f894-4822-a7dd-a0179a8f2046 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:45.085207 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-52c2ebb3-f894-4822-a7dd-a0179a8f2046 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:45.087405 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 874/1767] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:30:44 2022] DELETE /identity/v3/projects/386778dcad4d45c7bc5c343b735983b2 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:45.088573 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2ebb3-f894-4822-a7dd-a0179a8f2046 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:45.091559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-52c2ebb3-f894-4822-a7dd-a0179a8f2046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:45.128654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:45.204151 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2ebb3-f894-4822-a7dd-a0179a8f2046 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:45.214427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b7f40-f70f-4089-a1da-970dabadefae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:45.215284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b7f40-f70f-4089-a1da-970dabadefae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:45.215622 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b7f40-f70f-4089-a1da-970dabadefae admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:45.215965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b7f40-f70f-4089-a1da-970dabadefae admin admin] PATH_INFO: `/v3/users/6db1e2303fc64d8ea7e479dab966c62d` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:45.217891 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b7f40-f70f-4089-a1da-970dabadefae admin admin] RBAC: Authorizing `identity:delete_user(user_id=6db1e2303fc64d8ea7e479dab966c62d)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:45.219749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b7f40-f70f-4089-a1da-970dabadefae admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:45.225035 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dbd531-7715-4516-81d0-9333fbc607dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:45.225666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbd531-7715-4516-81d0-9333fbc607dd admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:45.225948 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbd531-7715-4516-81d0-9333fbc607dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:45.226221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbd531-7715-4516-81d0-9333fbc607dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:45.226909 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dbd531-7715-4516-81d0-9333fbc607dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:45.232336 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dbd531-7715-4516-81d0-9333fbc607dd admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:45.288340 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 894/1768] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:45 2022] POST /identity/v3/projects => generated 385 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:45.288340 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a19ca2-8e48-4731-99db-58dcc0c3c59b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:45.288340 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a19ca2-8e48-4731-99db-58dcc0c3c59b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:45.288340 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a19ca2-8e48-4731-99db-58dcc0c3c59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:45.298493 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fe7b7f40-f70f-4089-a1da-970dabadefae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6db1e2303fc64d8ea7e479dab966c62d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:45.307227 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fe7b7f40-f70f-4089-a1da-970dabadefae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6db1e2303fc64d8ea7e479dab966c62d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:45.310353 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f0a19ca2-8e48-4731-99db-58dcc0c3c59b None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:45.311620 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f0a19ca2-8e48-4731-99db-58dcc0c3c59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:45.325850 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fe7b7f40-f70f-4089-a1da-970dabadefae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6db1e2303fc64d8ea7e479dab966c62d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:45.326817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fe7b7f40-f70f-4089-a1da-970dabadefae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6db1e2303fc64d8ea7e479dab966c62d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:45.328063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fe7b7f40-f70f-4089-a1da-970dabadefae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6db1e2303fc64d8ea7e479dab966c62d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:45.344330 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 875/1769] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:30:45 2022] DELETE /identity/v3/users/6db1e2303fc64d8ea7e479dab966c62d => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:45.345541 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b7f40-f70f-4089-a1da-970dabadefae admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:45.347442 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fe7b7f40-f70f-4089-a1da-970dabadefae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:45.396531 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 895/1770] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:45.398456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769bcae-250e-49e4-bcf8-daf60f1c59a6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:45.398789 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769bcae-250e-49e4-bcf8-daf60f1c59a6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:45.399048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769bcae-250e-49e4-bcf8-daf60f1c59a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:45.425646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f769bcae-250e-49e4-bcf8-daf60f1c59a6 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:45.427258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f769bcae-250e-49e4-bcf8-daf60f1c59a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:45.436772 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b7f40-f70f-4089-a1da-970dabadefae admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:45.436973 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 896/1771] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:45.440604 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769bcae-250e-49e4-bcf8-daf60f1c59a6 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:45.440604 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f769bcae-250e-49e4-bcf8-daf60f1c59a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:45.446323 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092469fc-ff35-41a3-a89f-56be4413748d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:45.447082 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092469fc-ff35-41a3-a89f-56be4413748d admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:45.447448 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092469fc-ff35-41a3-a89f-56be4413748d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:45.447795 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092469fc-ff35-41a3-a89f-56be4413748d admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:45.448505 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092469fc-ff35-41a3-a89f-56be4413748d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:45.450762 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092469fc-ff35-41a3-a89f-56be4413748d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:45.452056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769bcae-250e-49e4-bcf8-daf60f1c59a6 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:45.457897 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30434dff-ad7e-4251-ad4e-88c38f0b7114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:45.458550 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30434dff-ad7e-4251-ad4e-88c38f0b7114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:45.458835 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30434dff-ad7e-4251-ad4e-88c38f0b7114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:45.459112 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30434dff-ad7e-4251-ad4e-88c38f0b7114 admin admin] PATH_INFO: `/v3/projects/a8b1fcc1df0b47d8937792e9d2d3b87c` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:45.460317 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30434dff-ad7e-4251-ad4e-88c38f0b7114 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8b1fcc1df0b47d8937792e9d2d3b87c)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:45.462490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30434dff-ad7e-4251-ad4e-88c38f0b7114 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:45.478989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 876/1772] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:45 2022] POST /identity/v3/users => generated 299 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:45.478989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402a39e-d368-4d91-9487-c8ebdb1d3dca None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:45.478989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402a39e-d368-4d91-9487-c8ebdb1d3dca None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:45.478989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402a39e-d368-4d91-9487-c8ebdb1d3dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:45.501309 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-a402a39e-d368-4d91-9487-c8ebdb1d3dca None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:45.502521 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a402a39e-d368-4d91-9487-c8ebdb1d3dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:45.516844 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-30434dff-ad7e-4251-ad4e-88c38f0b7114 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8b1fcc1df0b47d8937792e9d2d3b87c', 'request_id': 'req-30434dff-ad7e-4251-ad4e-88c38f0b7114'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:45.524334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-30434dff-ad7e-4251-ad4e-88c38f0b7114 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8b1fcc1df0b47d8937792e9d2d3b87c', 'request_id': 'req-30434dff-ad7e-4251-ad4e-88c38f0b7114'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:45.532408 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:45.533028 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-30434dff-ad7e-4251-ad4e-88c38f0b7114 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8b1fcc1df0b47d8937792e9d2d3b87c', 'request_id': 'req-30434dff-ad7e-4251-ad4e-88c38f0b7114'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:45.540590 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:45.545700 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-30434dff-ad7e-4251-ad4e-88c38f0b7114 admin admin] The token cache is being invalidate because project a8b1fcc1df0b47d8937792e9d2d3b87c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:30:45.546422 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-30434dff-ad7e-4251-ad4e-88c38f0b7114 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:45.547877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-30434dff-ad7e-4251-ad4e-88c38f0b7114 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:45.550050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 897/1773] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:30:45 2022] DELETE /identity/v3/projects/a8b1fcc1df0b47d8937792e9d2d3b87c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:45.551394 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30434dff-ad7e-4251-ad4e-88c38f0b7114 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:45.553430 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-30434dff-ad7e-4251-ad4e-88c38f0b7114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:45.584435 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 877/1774] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:45.585784 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402a39e-d368-4d91-9487-c8ebdb1d3dca None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:45.587295 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a402a39e-d368-4d91-9487-c8ebdb1d3dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:45.599095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402a39e-d368-4d91-9487-c8ebdb1d3dca None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:45.603807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581adf00-f440-4c1e-89b8-53f15dd58e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:45.604492 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581adf00-f440-4c1e-89b8-53f15dd58e29 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:45.604779 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581adf00-f440-4c1e-89b8-53f15dd58e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:45.605054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581adf00-f440-4c1e-89b8-53f15dd58e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:45.605774 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581adf00-f440-4c1e-89b8-53f15dd58e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:45.607384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581adf00-f440-4c1e-89b8-53f15dd58e29 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:45.614278 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30434dff-ad7e-4251-ad4e-88c38f0b7114 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:45.618196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b11a9-d873-4500-8248-b1cdb4908e4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:45.618458 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 878/1775] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:45.618975 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b11a9-d873-4500-8248-b1cdb4908e4a service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:45.619254 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b11a9-d873-4500-8248-b1cdb4908e4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:45.619527 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b11a9-d873-4500-8248-b1cdb4908e4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:45.619836 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f47ca-9595-42f4-8dde-fa33f313121f None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:45.620123 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f47ca-9595-42f4-8dde-fa33f313121f None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:45.620500 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f47ca-9595-42f4-8dde-fa33f313121f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:45.621433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e42b11a9-d873-4500-8248-b1cdb4908e4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:45.634515 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b11a9-d873-4500-8248-b1cdb4908e4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:45.635315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b11a9-d873-4500-8248-b1cdb4908e4a service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:45.640220 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ae1f47ca-9595-42f4-8dde-fa33f313121f None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:45.641457 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ae1f47ca-9595-42f4-8dde-fa33f313121f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:45.645306 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 898/1776] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:45.645959 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b11a9-d873-4500-8248-b1cdb4908e4a service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:45.649785 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 879/1777] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:45.650657 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f47ca-9595-42f4-8dde-fa33f313121f None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:45.655715 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b11a9-d873-4500-8248-b1cdb4908e4a service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:45.660489 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c600ac1-9aad-44b9-98ac-f9243985674a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:45.661124 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f47ca-9595-42f4-8dde-fa33f313121f None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:45.661306 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c600ac1-9aad-44b9-98ac-f9243985674a admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:45.661557 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c600ac1-9aad-44b9-98ac-f9243985674a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:45.661851 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c600ac1-9aad-44b9-98ac-f9243985674a admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:45.662555 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c600ac1-9aad-44b9-98ac-f9243985674a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:30:45.662882 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c600ac1-9aad-44b9-98ac-f9243985674a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:45.664527 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c600ac1-9aad-44b9-98ac-f9243985674a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:45.666482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04382b95-b85e-477f-b568-43ef27796781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:45.667246 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04382b95-b85e-477f-b568-43ef27796781 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:45.668582 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04382b95-b85e-477f-b568-43ef27796781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:45.669203 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04382b95-b85e-477f-b568-43ef27796781 admin admin] PATH_INFO: `/v3/projects/0c794f4a8a9248b0834dcc49d32b7c22/users/d86bd0504eb34d2da185e5cfa8332164/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:45.679664 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 899/1778] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:30:45.680749 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68eea5-d72b-4d9c-a4f3-ec694848fdeb None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:45.681031 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68eea5-d72b-4d9c-a4f3-ec694848fdeb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:45.681470 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68eea5-d72b-4d9c-a4f3-ec694848fdeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:45.692951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04382b95-b85e-477f-b568-43ef27796781 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c794f4a8a9248b0834dcc49d32b7c22, user_id=d86bd0504eb34d2da185e5cfa8332164, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:45.695811 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04382b95-b85e-477f-b568-43ef27796781 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:45.706358 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5e68eea5-d72b-4d9c-a4f3-ec694848fdeb None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:45.708241 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5e68eea5-d72b-4d9c-a4f3-ec694848fdeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:45.715750 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 880/1779] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:45 2022] PUT /identity/v3/projects/0c794f4a8a9248b0834dcc49d32b7c22/users/d86bd0504eb34d2da185e5cfa8332164/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:45.716508 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04382b95-b85e-477f-b568-43ef27796781 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:45.804542 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 900/1780] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:45.805689 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68eea5-d72b-4d9c-a4f3-ec694848fdeb None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:45.814010 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04382b95-b85e-477f-b568-43ef27796781 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:45.816747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68eea5-d72b-4d9c-a4f3-ec694848fdeb None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:45.822012 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69cdf96-03a3-41fa-ae7f-8dab6e57b9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:45.822878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cdf96-03a3-41fa-ae7f-8dab6e57b9a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:45.823436 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cdf96-03a3-41fa-ae7f-8dab6e57b9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:45.824024 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cdf96-03a3-41fa-ae7f-8dab6e57b9a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:45.824477 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6ae66-f0df-4612-bbf9-f0de4a9b86ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:45.825095 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69cdf96-03a3-41fa-ae7f-8dab6e57b9a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:45.826842 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69cdf96-03a3-41fa-ae7f-8dab6e57b9a1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:45.827333 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6ae66-f0df-4612-bbf9-f0de4a9b86ae admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:45.827698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6ae66-f0df-4612-bbf9-f0de4a9b86ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:45.830722 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6ae66-f0df-4612-bbf9-f0de4a9b86ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:45.831562 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6ae66-f0df-4612-bbf9-f0de4a9b86ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:45.834166 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6ae66-f0df-4612-bbf9-f0de4a9b86ae admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:45.844347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 901/1781] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:45.845591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69cdf96-03a3-41fa-ae7f-8dab6e57b9a1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:45.859270 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69cdf96-03a3-41fa-ae7f-8dab6e57b9a1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:45.865352 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650f9d4-73ed-47f7-9e70-e7e40fb574f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:45.866500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650f9d4-73ed-47f7-9e70-e7e40fb574f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:45.866994 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650f9d4-73ed-47f7-9e70-e7e40fb574f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:45.867519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650f9d4-73ed-47f7-9e70-e7e40fb574f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:45.868747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7650f9d4-73ed-47f7-9e70-e7e40fb574f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:30:45.869277 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7650f9d4-73ed-47f7-9e70-e7e40fb574f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:45.871999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7650f9d4-73ed-47f7-9e70-e7e40fb574f0 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:45.874170 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 881/1782] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:45 2022] POST /identity/v3/projects => generated 403 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:45.875096 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6ae66-f0df-4612-bbf9-f0de4a9b86ae admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:45.913150 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 902/1783] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:30:45.917270 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650f9d4-73ed-47f7-9e70-e7e40fb574f0 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:45.931777 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650f9d4-73ed-47f7-9e70-e7e40fb574f0 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:45.958285 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6ae66-f0df-4612-bbf9-f0de4a9b86ae admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:45.964914 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272629ef-a70b-4c78-b82a-62e6edafe77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:45.966111 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272629ef-a70b-4c78-b82a-62e6edafe77d admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:45.966536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272629ef-a70b-4c78-b82a-62e6edafe77d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:45.966970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272629ef-a70b-4c78-b82a-62e6edafe77d admin admin] PATH_INFO: `/v3/projects/0c794f4a8a9248b0834dcc49d32b7c22/users/d86bd0504eb34d2da185e5cfa8332164/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:45.969618 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272629ef-a70b-4c78-b82a-62e6edafe77d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c794f4a8a9248b0834dcc49d32b7c22, user_id=d86bd0504eb34d2da185e5cfa8332164, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:45.973222 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272629ef-a70b-4c78-b82a-62e6edafe77d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:45.987724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 882/1784] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:45 2022] PUT /identity/v3/projects/0c794f4a8a9248b0834dcc49d32b7c22/users/d86bd0504eb34d2da185e5cfa8332164/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:45.988906 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272629ef-a70b-4c78-b82a-62e6edafe77d admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:46.016588 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:46.028263 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028cc10-42a1-427c-bc68-06d6dcdc247f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:46.028940 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028cc10-42a1-427c-bc68-06d6dcdc247f admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:46.029538 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028cc10-42a1-427c-bc68-06d6dcdc247f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:46.029538 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028cc10-42a1-427c-bc68-06d6dcdc247f admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:46.030164 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028cc10-42a1-427c-bc68-06d6dcdc247f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:46.032086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028cc10-42a1-427c-bc68-06d6dcdc247f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:46.042291 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:46.058356 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 903/1785] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:45 2022] POST /identity/v3/users => generated 302 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:46.059450 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331dcbf-2c46-413a-9dda-45836247aeb2 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:46.059700 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331dcbf-2c46-413a-9dda-45836247aeb2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:46.059978 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331dcbf-2c46-413a-9dda-45836247aeb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:46.089409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272629ef-a70b-4c78-b82a-62e6edafe77d admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:46.095919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81102765-4c05-4539-b9d8-7aa91881e17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:46.096872 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81102765-4c05-4539-b9d8-7aa91881e17b admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:46.097090 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81102765-4c05-4539-b9d8-7aa91881e17b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:46.097163 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81102765-4c05-4539-b9d8-7aa91881e17b admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:46.098457 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81102765-4c05-4539-b9d8-7aa91881e17b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:46.100419 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4331dcbf-2c46-413a-9dda-45836247aeb2 None None] MFA Rules not processed for user `d86bd0504eb34d2da185e5cfa8332164`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:46.101640 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4331dcbf-2c46-413a-9dda-45836247aeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:46.101876 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81102765-4c05-4539-b9d8-7aa91881e17b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:46.134194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 883/1786] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:45 2022] POST /identity/v3/projects => generated 365 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:46.135131 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81102765-4c05-4539-b9d8-7aa91881e17b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:46.177563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 904/1787] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:46.178412 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331dcbf-2c46-413a-9dda-45836247aeb2 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:46.179679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4331dcbf-2c46-413a-9dda-45836247aeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:46.193912 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331dcbf-2c46-413a-9dda-45836247aeb2 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:46.198950 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ebb68-d674-4101-8c02-7b114e219a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:46.199527 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ebb68-d674-4101-8c02-7b114e219a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:46.199806 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ebb68-d674-4101-8c02-7b114e219a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:46.199896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81102765-4c05-4539-b9d8-7aa91881e17b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:46.200081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ebb68-d674-4101-8c02-7b114e219a3f admin admin] PATH_INFO: `/v3/projects/a30bc03cd15949cca57a8a810d820747` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:46.201277 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ebb68-d674-4101-8c02-7b114e219a3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a30bc03cd15949cca57a8a810d820747)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:46.203193 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ebb68-d674-4101-8c02-7b114e219a3f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:46.204631 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a3b33-9f55-40ca-9458-ab61f91c8f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:46.205254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a3b33-9f55-40ca-9458-ab61f91c8f40 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:46.205541 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a3b33-9f55-40ca-9458-ab61f91c8f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:46.205819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a3b33-9f55-40ca-9458-ab61f91c8f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:46.206547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a3b33-9f55-40ca-9458-ab61f91c8f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:46.208792 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a3b33-9f55-40ca-9458-ab61f91c8f40 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:46.218882 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 884/1788] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:46.219674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a3b33-9f55-40ca-9458-ab61f91c8f40 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:46.229073 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a3b33-9f55-40ca-9458-ab61f91c8f40 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:46.233925 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d76eb9-afab-4b5a-ba0e-efe900cb4aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:46.234546 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d76eb9-afab-4b5a-ba0e-efe900cb4aff admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:46.234827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d76eb9-afab-4b5a-ba0e-efe900cb4aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:46.235099 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d76eb9-afab-4b5a-ba0e-efe900cb4aff admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:46.235753 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d76eb9-afab-4b5a-ba0e-efe900cb4aff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:46.237684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d76eb9-afab-4b5a-ba0e-efe900cb4aff admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:46.253537 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-257ebb68-d674-4101-8c02-7b114e219a3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a30bc03cd15949cca57a8a810d820747', 'request_id': 'req-257ebb68-d674-4101-8c02-7b114e219a3f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:46.259872 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-257ebb68-d674-4101-8c02-7b114e219a3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a30bc03cd15949cca57a8a810d820747', 'request_id': 'req-257ebb68-d674-4101-8c02-7b114e219a3f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:46.260820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 885/1789] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:46 2022] POST /identity/v3/users => generated 283 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:46.261631 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d76eb9-afab-4b5a-ba0e-efe900cb4aff admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:46.263200 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-77d76eb9-afab-4b5a-ba0e-efe900cb4aff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:46.269938 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-257ebb68-d674-4101-8c02-7b114e219a3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a30bc03cd15949cca57a8a810d820747', 'request_id': 'req-257ebb68-d674-4101-8c02-7b114e219a3f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:46.281661 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-257ebb68-d674-4101-8c02-7b114e219a3f admin admin] The token cache is being invalidate because project a30bc03cd15949cca57a8a810d820747 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:30:46.281956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-257ebb68-d674-4101-8c02-7b114e219a3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:46.282616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-257ebb68-d674-4101-8c02-7b114e219a3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:46.284218 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 905/1790] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:30:46 2022] DELETE /identity/v3/projects/a30bc03cd15949cca57a8a810d820747 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:46.285392 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ebb68-d674-4101-8c02-7b114e219a3f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:46.286349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-257ebb68-d674-4101-8c02-7b114e219a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:46.330553 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d76eb9-afab-4b5a-ba0e-efe900cb4aff admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:46.353768 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ebb68-d674-4101-8c02-7b114e219a3f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:46.359422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:46.359776 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f5a9b-0deb-49e1-8f11-e53572f1f8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:46.360593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f5a9b-0deb-49e1-8f11-e53572f1f8c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:46.361234 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f5a9b-0deb-49e1-8f11-e53572f1f8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:46.361642 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f5a9b-0deb-49e1-8f11-e53572f1f8c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:46.362572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f5a9b-0deb-49e1-8f11-e53572f1f8c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:46.364818 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f5a9b-0deb-49e1-8f11-e53572f1f8c3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:46.367419 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:46.382008 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 906/1791] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:46.383091 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f5a9b-0deb-49e1-8f11-e53572f1f8c3 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:46.387862 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7c5f5a9b-0deb-49e1-8f11-e53572f1f8c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:46.390741 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc44f1-9632-4327-88c7-5624630a25d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:46.391454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc44f1-9632-4327-88c7-5624630a25d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:46.391659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc44f1-9632-4327-88c7-5624630a25d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:46.391939 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc44f1-9632-4327-88c7-5624630a25d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:46.393757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f7dc44f1-9632-4327-88c7-5624630a25d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:46.402080 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f5a9b-0deb-49e1-8f11-e53572f1f8c3 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:46.406795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76552f-9aa3-40a7-a675-7507674913fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:46.407436 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76552f-9aa3-40a7-a675-7507674913fd admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:46.407710 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76552f-9aa3-40a7-a675-7507674913fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:46.407983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76552f-9aa3-40a7-a675-7507674913fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:46.408713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76552f-9aa3-40a7-a675-7507674913fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:46.410282 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76552f-9aa3-40a7-a675-7507674913fd admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:46.424674 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 907/1792] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:46.426999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd8853-8084-47b7-826a-ac544b2e5d00 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:46.427512 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd8853-8084-47b7-826a-ac544b2e5d00 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:46.428010 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd8853-8084-47b7-826a-ac544b2e5d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:46.450050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f2dd8853-8084-47b7-826a-ac544b2e5d00 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:46.451432 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f2dd8853-8084-47b7-826a-ac544b2e5d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:46.460273 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 908/1793] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:46.461034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd8853-8084-47b7-826a-ac544b2e5d00 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:46.471042 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd8853-8084-47b7-826a-ac544b2e5d00 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:46.472232 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc44f1-9632-4327-88c7-5624630a25d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:46.473900 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc44f1-9632-4327-88c7-5624630a25d1 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:46.480592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb19157-fe6c-4b78-960e-86367f56f7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:46.481330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb19157-fe6c-4b78-960e-86367f56f7e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:46.481616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb19157-fe6c-4b78-960e-86367f56f7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:46.481887 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb19157-fe6c-4b78-960e-86367f56f7e4 admin admin] PATH_INFO: `/v3/projects/3997142544354d13a045c4765fe1fb49/users/1f11ddea61374b0dbf660d699cb9b121/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:46.487586 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 886/1794] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:46.489246 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc44f1-9632-4327-88c7-5624630a25d1 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:46.498003 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb19157-fe6c-4b78-960e-86367f56f7e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3997142544354d13a045c4765fe1fb49, user_id=1f11ddea61374b0dbf660d699cb9b121, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:46.499318 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc44f1-9632-4327-88c7-5624630a25d1 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:46.500868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb19157-fe6c-4b78-960e-86367f56f7e4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:46.505288 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804582a-cb48-4971-b90a-441b4906a996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:46.505938 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804582a-cb48-4971-b90a-441b4906a996 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:46.506223 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804582a-cb48-4971-b90a-441b4906a996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:46.506526 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804582a-cb48-4971-b90a-441b4906a996 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:46.507355 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804582a-cb48-4971-b90a-441b4906a996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:46.513274 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804582a-cb48-4971-b90a-441b4906a996 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:46.517089 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 909/1795] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:46 2022] PUT /identity/v3/projects/3997142544354d13a045c4765fe1fb49/users/1f11ddea61374b0dbf660d699cb9b121/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:46.518395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9de4e0-d073-4045-9585-0caea48eec24 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:46.518684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9de4e0-d073-4045-9585-0caea48eec24 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:46.519142 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9de4e0-d073-4045-9585-0caea48eec24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:46.527307 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 887/1796] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:46.529450 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804582a-cb48-4971-b90a-441b4906a996 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:46.531614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4804582a-cb48-4971-b90a-441b4906a996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:46.540101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8b9de4e0-d073-4045-9585-0caea48eec24 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:46.541351 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8b9de4e0-d073-4045-9585-0caea48eec24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:46.544566 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804582a-cb48-4971-b90a-441b4906a996 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:46.601235 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 910/1797] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:46.602511 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e544e22-6d34-4013-ab43-b5c2e595329f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:46.602801 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e544e22-6d34-4013-ab43-b5c2e595329f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:46.603082 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e544e22-6d34-4013-ab43-b5c2e595329f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:46.607070 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fda5b2-1461-48ff-9e65-f40635e5e1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:46.607717 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fda5b2-1461-48ff-9e65-f40635e5e1e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:46.608121 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fda5b2-1461-48ff-9e65-f40635e5e1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:46.609062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fda5b2-1461-48ff-9e65-f40635e5e1e7 admin admin] PATH_INFO: `/v3/users/5cd8417063fd49cbb5488c16501853ca` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:46.610258 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fda5b2-1461-48ff-9e65-f40635e5e1e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cd8417063fd49cbb5488c16501853ca)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:46.612473 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fda5b2-1461-48ff-9e65-f40635e5e1e7 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:46.624115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4e544e22-6d34-4013-ab43-b5c2e595329f None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:46.625370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4e544e22-6d34-4013-ab43-b5c2e595329f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:46.637040 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 911/1798] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:46.637040 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e544e22-6d34-4013-ab43-b5c2e595329f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:46.653606 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e544e22-6d34-4013-ab43-b5c2e595329f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:46.658857 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e72c07-dfcb-42bf-ad76-e72e76d83270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:46.659431 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e72c07-dfcb-42bf-ad76-e72e76d83270 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:46.659720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e72c07-dfcb-42bf-ad76-e72e76d83270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:46.660143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e72c07-dfcb-42bf-ad76-e72e76d83270 admin admin] PATH_INFO: `/v3/projects/832ae09639224ab1834e30e816f7d09f/users/23a86fdf37434ee9af60be66d66b22af/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:46.671432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-77fda5b2-1461-48ff-9e65-f40635e5e1e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cd8417063fd49cbb5488c16501853ca'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:46.676717 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e72c07-dfcb-42bf-ad76-e72e76d83270 admin admin] RBAC: Authorizing `identity:create_grant(project_id=832ae09639224ab1834e30e816f7d09f, user_id=23a86fdf37434ee9af60be66d66b22af, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:46.679407 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e72c07-dfcb-42bf-ad76-e72e76d83270 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:46.679913 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-77fda5b2-1461-48ff-9e65-f40635e5e1e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cd8417063fd49cbb5488c16501853ca'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:46.693399 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:46.694771 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 912/1799] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:46 2022] PUT /identity/v3/projects/832ae09639224ab1834e30e816f7d09f/users/23a86fdf37434ee9af60be66d66b22af/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:46.695498 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e72c07-dfcb-42bf-ad76-e72e76d83270 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:46.696962 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-77fda5b2-1461-48ff-9e65-f40635e5e1e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cd8417063fd49cbb5488c16501853ca'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:46.699275 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-77fda5b2-1461-48ff-9e65-f40635e5e1e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cd8417063fd49cbb5488c16501853ca'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:46.703921 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-77fda5b2-1461-48ff-9e65-f40635e5e1e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cd8417063fd49cbb5488c16501853ca'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:46.719775 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 888/1800] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:30:46 2022] DELETE /identity/v3/users/5cd8417063fd49cbb5488c16501853ca => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:46.721218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fda5b2-1461-48ff-9e65-f40635e5e1e7 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:46.722943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-77fda5b2-1461-48ff-9e65-f40635e5e1e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:46.769241 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e72c07-dfcb-42bf-ad76-e72e76d83270 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:46.792295 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fda5b2-1461-48ff-9e65-f40635e5e1e7 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:46.796468 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b9eaf-5472-426b-99a2-39fee215aa51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:46.797085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b9eaf-5472-426b-99a2-39fee215aa51 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:46.797366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b9eaf-5472-426b-99a2-39fee215aa51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:46.797643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b9eaf-5472-426b-99a2-39fee215aa51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:46.799424 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-235b9eaf-5472-426b-99a2-39fee215aa51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:46.799575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:46.828015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae1041-6bd6-461f-95e0-9b6edcb01f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:46.828675 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae1041-6bd6-461f-95e0-9b6edcb01f5c admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:46.828961 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae1041-6bd6-461f-95e0-9b6edcb01f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:46.829237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae1041-6bd6-461f-95e0-9b6edcb01f5c admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:46.829933 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae1041-6bd6-461f-95e0-9b6edcb01f5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:30:46.830237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae1041-6bd6-461f-95e0-9b6edcb01f5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:46.831858 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae1041-6bd6-461f-95e0-9b6edcb01f5c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:46.843514 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 913/1801] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:30:46.845153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873093a-70d1-48cd-b0c8-0141327a1332 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:46.845449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873093a-70d1-48cd-b0c8-0141327a1332 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:46.845729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873093a-70d1-48cd-b0c8-0141327a1332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:46.862726 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b9eaf-5472-426b-99a2-39fee215aa51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:46.864262 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b9eaf-5472-426b-99a2-39fee215aa51 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:46.873242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8873093a-70d1-48cd-b0c8-0141327a1332 None None] MFA Rules not processed for user `23a86fdf37434ee9af60be66d66b22af`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:46.874448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8873093a-70d1-48cd-b0c8-0141327a1332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:46.874580 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 889/1802] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:46.875858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664fa25-714b-4114-becd-740553cd2cd6 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:46.876152 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664fa25-714b-4114-becd-740553cd2cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:46.876464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664fa25-714b-4114-becd-740553cd2cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:46.897538 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-4664fa25-714b-4114-becd-740553cd2cd6 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:46.898631 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4664fa25-714b-4114-becd-740553cd2cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:46.912994 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 890/1803] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:46.912994 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664fa25-714b-4114-becd-740553cd2cd6 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:46.914639 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4664fa25-714b-4114-becd-740553cd2cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:46.926689 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664fa25-714b-4114-becd-740553cd2cd6 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:46.931892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a53885-6abe-43c2-8831-92704c29decb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:46.932571 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a53885-6abe-43c2-8831-92704c29decb admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:46.932880 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a53885-6abe-43c2-8831-92704c29decb admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:46.933191 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a53885-6abe-43c2-8831-92704c29decb admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:46.933894 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a53885-6abe-43c2-8831-92704c29decb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:46.935857 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a53885-6abe-43c2-8831-92704c29decb admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:46.962088 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 914/1804] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:46.963202 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873093a-70d1-48cd-b0c8-0141327a1332 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:46.964874 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8873093a-70d1-48cd-b0c8-0141327a1332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:46.973374 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 891/1805] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:46 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:46.974175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a53885-6abe-43c2-8831-92704c29decb admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:46.976559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-01a53885-6abe-43c2-8831-92704c29decb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:46.980842 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873093a-70d1-48cd-b0c8-0141327a1332 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:47.040235 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0cafa-3b5b-450b-b3af-62a276e64b74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:47.040898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0cafa-3b5b-450b-b3af-62a276e64b74 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:47.041186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0cafa-3b5b-450b-b3af-62a276e64b74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:47.041458 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0cafa-3b5b-450b-b3af-62a276e64b74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:47.101703 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a53885-6abe-43c2-8831-92704c29decb admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:47.111028 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0cafa-3b5b-450b-b3af-62a276e64b74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:47.112090 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868725b-9d87-4d19-b9d2-49f6fdc4ebf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:47.112562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0cafa-3b5b-450b-b3af-62a276e64b74 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:47.113057 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868725b-9d87-4d19-b9d2-49f6fdc4ebf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:47.113869 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868725b-9d87-4d19-b9d2-49f6fdc4ebf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:47.114149 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868725b-9d87-4d19-b9d2-49f6fdc4ebf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:47.115929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868725b-9d87-4d19-b9d2-49f6fdc4ebf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:47.118039 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868725b-9d87-4d19-b9d2-49f6fdc4ebf5 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:47.124132 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 915/1806] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:47.125031 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0cafa-3b5b-450b-b3af-62a276e64b74 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:47.135327 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0cafa-3b5b-450b-b3af-62a276e64b74 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:47.139170 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:47.140256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5f107-ef4e-469e-82e9-b4b3325e2eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:47.141024 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5f107-ef4e-469e-82e9-b4b3325e2eac admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:47.141599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5f107-ef4e-469e-82e9-b4b3325e2eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:47.141767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5f107-ef4e-469e-82e9-b4b3325e2eac admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:47.142462 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5f107-ef4e-469e-82e9-b4b3325e2eac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:47.144489 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5f107-ef4e-469e-82e9-b4b3325e2eac admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:47.155170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:47.161041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 892/1807] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:46 2022] POST /identity/v3/projects => generated 363 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:47.162577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868725b-9d87-4d19-b9d2-49f6fdc4ebf5 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:47.173234 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 916/1808] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:47 2022] POST /identity/v3/users => generated 289 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:47.181711 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5f107-ef4e-469e-82e9-b4b3325e2eac admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:47.192290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5f107-ef4e-469e-82e9-b4b3325e2eac admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:47.243753 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868725b-9d87-4d19-b9d2-49f6fdc4ebf5 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:47.249685 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b03a94-77cb-40f6-8910-40e033a7ced0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:47.250415 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b03a94-77cb-40f6-8910-40e033a7ced0 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:47.250993 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b03a94-77cb-40f6-8910-40e033a7ced0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:47.251359 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b03a94-77cb-40f6-8910-40e033a7ced0 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:47.252407 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b03a94-77cb-40f6-8910-40e033a7ced0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:47.255006 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b03a94-77cb-40f6-8910-40e033a7ced0 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:47.267963 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7796ce-2f41-4186-9e22-6e6b2d61b798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:47.270048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7796ce-2f41-4186-9e22-6e6b2d61b798 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:47.270541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7796ce-2f41-4186-9e22-6e6b2d61b798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:47.271116 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7796ce-2f41-4186-9e22-6e6b2d61b798 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:47.272330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7796ce-2f41-4186-9e22-6e6b2d61b798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:47.274923 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7796ce-2f41-4186-9e22-6e6b2d61b798 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:47.284757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 893/1809] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:47 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:47.293305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 917/1810] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:47.296351 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7796ce-2f41-4186-9e22-6e6b2d61b798 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:47.297865 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b03a94-77cb-40f6-8910-40e033a7ced0 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:47.310135 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7796ce-2f41-4186-9e22-6e6b2d61b798 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:47.318137 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b03a94-77cb-40f6-8910-40e033a7ced0 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:47.319792 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a1239-d2de-4078-9f9c-5a01644dabfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:47.320569 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a1239-d2de-4078-9f9c-5a01644dabfa admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:47.320858 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a1239-d2de-4078-9f9c-5a01644dabfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:47.321141 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a1239-d2de-4078-9f9c-5a01644dabfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:47.322829 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7a1239-d2de-4078-9f9c-5a01644dabfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:47.325749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94909d60-1b01-4ef4-b4d3-5fc3ac594f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:47.326027 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7a1239-d2de-4078-9f9c-5a01644dabfa admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:47.326853 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94909d60-1b01-4ef4-b4d3-5fc3ac594f46 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:47.327611 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94909d60-1b01-4ef4-b4d3-5fc3ac594f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:47.328006 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94909d60-1b01-4ef4-b4d3-5fc3ac594f46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:47.328830 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94909d60-1b01-4ef4-b4d3-5fc3ac594f46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:47.330610 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94909d60-1b01-4ef4-b4d3-5fc3ac594f46 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:47.343739 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 918/1811] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:47.349670 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 894/1812] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:47.350729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94909d60-1b01-4ef4-b4d3-5fc3ac594f46 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:47.351713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a1239-d2de-4078-9f9c-5a01644dabfa admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:47.362103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94909d60-1b01-4ef4-b4d3-5fc3ac594f46 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:47.370524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a1239-d2de-4078-9f9c-5a01644dabfa admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:47.372763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f3e4b8-9738-43b6-af41-44f9640dec0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:47.373501 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3e4b8-9738-43b6-af41-44f9640dec0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:47.373729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3e4b8-9738-43b6-af41-44f9640dec0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:47.374473 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3e4b8-9738-43b6-af41-44f9640dec0e admin admin] PATH_INFO: `/v3/projects/94a888bc2de340b08d6ffd860e11b215/users/2fcacf154a864a2e921864e23dcebfd8/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:47.383077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d111e3-76bc-4e7e-8f79-5e271bc29e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:47.384292 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d111e3-76bc-4e7e-8f79-5e271bc29e60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:47.384621 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d111e3-76bc-4e7e-8f79-5e271bc29e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:47.384947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d111e3-76bc-4e7e-8f79-5e271bc29e60 admin admin] PATH_INFO: `/v3/projects/35d430f3c7484eeda44df70babe6c4da/users/a80b476746a34d98826c4290cb861501/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:47.405770 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f3e4b8-9738-43b6-af41-44f9640dec0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=94a888bc2de340b08d6ffd860e11b215, user_id=2fcacf154a864a2e921864e23dcebfd8, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:47.412409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f3e4b8-9738-43b6-af41-44f9640dec0e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:47.443484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d111e3-76bc-4e7e-8f79-5e271bc29e60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35d430f3c7484eeda44df70babe6c4da, user_id=a80b476746a34d98826c4290cb861501, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:47.453034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d111e3-76bc-4e7e-8f79-5e271bc29e60 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:47.464443 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 895/1813] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:47 2022] PUT /identity/v3/projects/94a888bc2de340b08d6ffd860e11b215/users/2fcacf154a864a2e921864e23dcebfd8/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:47.466075 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1eb43-74e9-464f-bf91-439a0cc1de58 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:47.466464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1eb43-74e9-464f-bf91-439a0cc1de58 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:47.466648 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1eb43-74e9-464f-bf91-439a0cc1de58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:47.481416 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 919/1814] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:47 2022] PUT /identity/v3/projects/35d430f3c7484eeda44df70babe6c4da/users/a80b476746a34d98826c4290cb861501/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:47.485546 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d111e3-76bc-4e7e-8f79-5e271bc29e60 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:47.508115 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-45a1eb43-74e9-464f-bf91-439a0cc1de58 None None] MFA Rules not processed for user `2fcacf154a864a2e921864e23dcebfd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:47.509380 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-45a1eb43-74e9-464f-bf91-439a0cc1de58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:47.599796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 896/1815] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:30:47.601268 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1eb43-74e9-464f-bf91-439a0cc1de58 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:47.642594 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d111e3-76bc-4e7e-8f79-5e271bc29e60 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:47.652544 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfc0ef-0227-47bd-b80d-4741750807f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:47.653669 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfc0ef-0227-47bd-b80d-4741750807f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:47.653669 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfc0ef-0227-47bd-b80d-4741750807f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:47.653954 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfc0ef-0227-47bd-b80d-4741750807f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:47.656639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfc0ef-0227-47bd-b80d-4741750807f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:47.658257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfc0ef-0227-47bd-b80d-4741750807f7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:47.669965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:47.671561 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 920/1816] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 12:30:47.676171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfc0ef-0227-47bd-b80d-4741750807f7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:47.692399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfc0ef-0227-47bd-b80d-4741750807f7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:47.698453 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bf6f6-2bb5-4abe-83ec-498cd3e758f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:47.700503 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bf6f6-2bb5-4abe-83ec-498cd3e758f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:47.701187 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bf6f6-2bb5-4abe-83ec-498cd3e758f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:47.701187 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bf6f6-2bb5-4abe-83ec-498cd3e758f3 admin admin] PATH_INFO: `/v3/projects/35d430f3c7484eeda44df70babe6c4da/users/a80b476746a34d98826c4290cb861501/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:47.704696 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bf6f6-2bb5-4abe-83ec-498cd3e758f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35d430f3c7484eeda44df70babe6c4da, user_id=a80b476746a34d98826c4290cb861501, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:47.707938 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bf6f6-2bb5-4abe-83ec-498cd3e758f3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:47.710041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1eb43-74e9-464f-bf91-439a0cc1de58 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:47.724389 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 921/1817] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:47 2022] PUT /identity/v3/projects/35d430f3c7484eeda44df70babe6c4da/users/a80b476746a34d98826c4290cb861501/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:47.732768 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc3fb8-88f8-42d4-805d-a4ecebad0182 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:47.733116 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc3fb8-88f8-42d4-805d-a4ecebad0182 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:47.733350 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc3fb8-88f8-42d4-805d-a4ecebad0182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:47.774239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-42dc3fb8-88f8-42d4-805d-a4ecebad0182 None None] MFA Rules not processed for user `a80b476746a34d98826c4290cb861501`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:47.776115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-42dc3fb8-88f8-42d4-805d-a4ecebad0182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:47.811916 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:47.877978 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f5d658-f26b-4132-96b9-ff26f1323c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:47.878671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5d658-f26b-4132-96b9-ff26f1323c6d admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:47.879893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5d658-f26b-4132-96b9-ff26f1323c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:47.879893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5d658-f26b-4132-96b9-ff26f1323c6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:47.880571 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f5d658-f26b-4132-96b9-ff26f1323c6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:47.881400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 922/1818] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:30:47.882447 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc3fb8-88f8-42d4-805d-a4ecebad0182 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:47.882742 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f5d658-f26b-4132-96b9-ff26f1323c6d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:47.883884 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-42dc3fb8-88f8-42d4-805d-a4ecebad0182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:47.926549 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc3fb8-88f8-42d4-805d-a4ecebad0182 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:47.940923 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab079ce-32a8-42ae-b6aa-1d8945375630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:47.951041 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab079ce-32a8-42ae-b6aa-1d8945375630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:47.951416 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab079ce-32a8-42ae-b6aa-1d8945375630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:47.951760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab079ce-32a8-42ae-b6aa-1d8945375630 admin admin] PATH_INFO: `/v3/projects/712607f8374e4c6fa783c34f0d59a45b` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:47.958528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab079ce-32a8-42ae-b6aa-1d8945375630 admin admin] RBAC: Authorizing `identity:delete_project(project_id=712607f8374e4c6fa783c34f0d59a45b)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:47.960723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab079ce-32a8-42ae-b6aa-1d8945375630 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:48.011347 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 897/1819] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:47 2022] POST /identity/v3/projects => generated 377 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:48.015586 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f5d658-f26b-4132-96b9-ff26f1323c6d admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:48.017340 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c1f5d658-f26b-4132-96b9-ff26f1323c6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:48.035305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bab079ce-32a8-42ae-b6aa-1d8945375630 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '712607f8374e4c6fa783c34f0d59a45b', 'request_id': 'req-bab079ce-32a8-42ae-b6aa-1d8945375630'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:48.042067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bab079ce-32a8-42ae-b6aa-1d8945375630 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '712607f8374e4c6fa783c34f0d59a45b', 'request_id': 'req-bab079ce-32a8-42ae-b6aa-1d8945375630'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:48.063808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bab079ce-32a8-42ae-b6aa-1d8945375630 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '712607f8374e4c6fa783c34f0d59a45b', 'request_id': 'req-bab079ce-32a8-42ae-b6aa-1d8945375630'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:48.087349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bab079ce-32a8-42ae-b6aa-1d8945375630 admin admin] The token cache is being invalidate because project 712607f8374e4c6fa783c34f0d59a45b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:30:48.087636 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bab079ce-32a8-42ae-b6aa-1d8945375630 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:48.088526 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bab079ce-32a8-42ae-b6aa-1d8945375630 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:48.093599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 923/1820] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:30:47 2022] DELETE /identity/v3/projects/712607f8374e4c6fa783c34f0d59a45b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:48.094420 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab079ce-32a8-42ae-b6aa-1d8945375630 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:48.096588 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bab079ce-32a8-42ae-b6aa-1d8945375630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:48.096731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f5d658-f26b-4132-96b9-ff26f1323c6d admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:48.167636 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709efe54-9ea1-429a-b587-25683ed7a394 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:48.169424 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709efe54-9ea1-429a-b587-25683ed7a394 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:48.169868 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709efe54-9ea1-429a-b587-25683ed7a394 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:48.170195 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709efe54-9ea1-429a-b587-25683ed7a394 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:48.172005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-709efe54-9ea1-429a-b587-25683ed7a394 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:48.183368 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab079ce-32a8-42ae-b6aa-1d8945375630 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:48.189205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed752742-7ce4-4e59-9ade-e48ae77c39f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:48.190069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed752742-7ce4-4e59-9ade-e48ae77c39f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:48.190423 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed752742-7ce4-4e59-9ade-e48ae77c39f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:48.190799 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed752742-7ce4-4e59-9ade-e48ae77c39f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:48.191597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed752742-7ce4-4e59-9ade-e48ae77c39f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:48.193693 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed752742-7ce4-4e59-9ade-e48ae77c39f7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:48.220380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 924/1821] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:48 2022] POST /identity/v3/users => generated 289 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:48.220380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed752742-7ce4-4e59-9ade-e48ae77c39f7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:48.221384 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ed752742-7ce4-4e59-9ade-e48ae77c39f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:48.224618 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:48.234571 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed752742-7ce4-4e59-9ade-e48ae77c39f7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:48.240091 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e61e6-084c-47f5-9535-b387910fe3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:48.241054 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e61e6-084c-47f5-9535-b387910fe3ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:48.241333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e61e6-084c-47f5-9535-b387910fe3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:48.241604 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e61e6-084c-47f5-9535-b387910fe3ee admin admin] PATH_INFO: `/v3/users/d86bd0504eb34d2da185e5cfa8332164` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:48.242814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e61e6-084c-47f5-9535-b387910fe3ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=d86bd0504eb34d2da185e5cfa8332164)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:48.244971 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e61e6-084c-47f5-9535-b387910fe3ee admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:48.252226 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709efe54-9ea1-429a-b587-25683ed7a394 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:48.254023 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709efe54-9ea1-429a-b587-25683ed7a394 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:48.265704 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 898/1822] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:48 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:48.266917 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff090232-3cf8-4d3e-98ed-f633eece41cf None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:48.267286 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff090232-3cf8-4d3e-98ed-f633eece41cf None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:48.267657 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff090232-3cf8-4d3e-98ed-f633eece41cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:48.277932 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:48.314357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-604e61e6-084c-47f5-9535-b387910fe3ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd86bd0504eb34d2da185e5cfa8332164'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:48.324608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ff090232-3cf8-4d3e-98ed-f633eece41cf None None] MFA Rules not processed for user `1f11ddea61374b0dbf660d699cb9b121`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:48.326039 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ff090232-3cf8-4d3e-98ed-f633eece41cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:48.333947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-604e61e6-084c-47f5-9535-b387910fe3ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd86bd0504eb34d2da185e5cfa8332164'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:48.335321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-604e61e6-084c-47f5-9535-b387910fe3ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd86bd0504eb34d2da185e5cfa8332164'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:48.336254 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-604e61e6-084c-47f5-9535-b387910fe3ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd86bd0504eb34d2da185e5cfa8332164'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:48.352062 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-604e61e6-084c-47f5-9535-b387910fe3ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd86bd0504eb34d2da185e5cfa8332164'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:48.382664 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 925/1823] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:30:48 2022] DELETE /identity/v3/users/d86bd0504eb34d2da185e5cfa8332164 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:48.383571 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e61e6-084c-47f5-9535-b387910fe3ee admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:48.385091 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-604e61e6-084c-47f5-9535-b387910fe3ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:48.441851 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 899/1824] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:48 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:48.442876 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff090232-3cf8-4d3e-98ed-f633eece41cf None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:48.447311 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ff090232-3cf8-4d3e-98ed-f633eece41cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:48.485381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e61e6-084c-47f5-9535-b387910fe3ee admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:48.491514 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbac743-efa7-44fa-847c-0135f3a67fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:48.492154 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbac743-efa7-44fa-847c-0135f3a67fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:48.492629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbac743-efa7-44fa-847c-0135f3a67fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:48.492997 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbac743-efa7-44fa-847c-0135f3a67fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:48.493613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbac743-efa7-44fa-847c-0135f3a67fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:48.495233 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbac743-efa7-44fa-847c-0135f3a67fc8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:48.496662 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff090232-3cf8-4d3e-98ed-f633eece41cf None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:48.510524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 926/1825] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:48.510524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbac743-efa7-44fa-847c-0135f3a67fc8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:48.512227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cfbac743-efa7-44fa-847c-0135f3a67fc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:48.526226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbac743-efa7-44fa-847c-0135f3a67fc8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:48.531689 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275a7bf-3aea-4f2d-9fca-8d74f38bf5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:48.532453 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275a7bf-3aea-4f2d-9fca-8d74f38bf5b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:48.532807 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275a7bf-3aea-4f2d-9fca-8d74f38bf5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:48.533152 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275a7bf-3aea-4f2d-9fca-8d74f38bf5b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:48.534018 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275a7bf-3aea-4f2d-9fca-8d74f38bf5b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:48.537285 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275a7bf-3aea-4f2d-9fca-8d74f38bf5b3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:48.577360 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 927/1826] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:48 2022] POST /identity/v3/projects => generated 405 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:48.578582 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275a7bf-3aea-4f2d-9fca-8d74f38bf5b3 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:48.636895 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39448ac-e05e-4ec1-9641-84387c6bbea7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:48.637601 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39448ac-e05e-4ec1-9641-84387c6bbea7 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:48.637952 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39448ac-e05e-4ec1-9641-84387c6bbea7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:48.638292 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39448ac-e05e-4ec1-9641-84387c6bbea7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:48.640283 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b39448ac-e05e-4ec1-9641-84387c6bbea7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:48.647961 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275a7bf-3aea-4f2d-9fca-8d74f38bf5b3 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:48.653205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c99f4c-9f05-45b1-930c-0431759b0484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:48.653829 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c99f4c-9f05-45b1-930c-0431759b0484 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:48.654113 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c99f4c-9f05-45b1-930c-0431759b0484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:48.654386 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c99f4c-9f05-45b1-930c-0431759b0484 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:48.655120 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c99f4c-9f05-45b1-930c-0431759b0484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:48.656889 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c99f4c-9f05-45b1-930c-0431759b0484 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:48.668464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 928/1827] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:48.669577 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c99f4c-9f05-45b1-930c-0431759b0484 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:48.683694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c99f4c-9f05-45b1-930c-0431759b0484 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:48.689171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1ff0b-292b-4106-bd4a-28d4851a77db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:48.689793 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1ff0b-292b-4106-bd4a-28d4851a77db admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:48.690083 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1ff0b-292b-4106-bd4a-28d4851a77db admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:48.690363 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1ff0b-292b-4106-bd4a-28d4851a77db admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:48.691032 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1ff0b-292b-4106-bd4a-28d4851a77db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:48.693458 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1ff0b-292b-4106-bd4a-28d4851a77db admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:48.711074 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:48.713600 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39448ac-e05e-4ec1-9641-84387c6bbea7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:48.715394 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39448ac-e05e-4ec1-9641-84387c6bbea7 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:48.725984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 929/1828] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:48 2022] POST /identity/v3/users => generated 309 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:48.727102 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 900/1829] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:48.727438 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1ff0b-292b-4106-bd4a-28d4851a77db admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:48.730077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39448ac-e05e-4ec1-9641-84387c6bbea7 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:48.739863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1ff0b-292b-4106-bd4a-28d4851a77db admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:48.744805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39448ac-e05e-4ec1-9641-84387c6bbea7 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:48.745455 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c96f0-1143-43ca-abf0-cc6f0c8ade02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:48.746133 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c96f0-1143-43ca-abf0-cc6f0c8ade02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:48.746421 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c96f0-1143-43ca-abf0-cc6f0c8ade02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:48.747258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c96f0-1143-43ca-abf0-cc6f0c8ade02 admin admin] PATH_INFO: `/v3/projects/4c250878bd204cb49f4458ace310ae9f/users/96014290ba4049ad85def453315128df/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:48.751370 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53a709-686c-42ba-9846-0af94245193b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:48.752079 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53a709-686c-42ba-9846-0af94245193b admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:48.752570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53a709-686c-42ba-9846-0af94245193b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:48.752922 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53a709-686c-42ba-9846-0af94245193b admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:48.754282 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53a709-686c-42ba-9846-0af94245193b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:48.755521 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53a709-686c-42ba-9846-0af94245193b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:48.768714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 901/1830] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 12:30:48.769058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c96f0-1143-43ca-abf0-cc6f0c8ade02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c250878bd204cb49f4458ace310ae9f, user_id=96014290ba4049ad85def453315128df, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:48.772778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c96f0-1143-43ca-abf0-cc6f0c8ade02 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:48.774172 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53a709-686c-42ba-9846-0af94245193b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:48.788243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53a709-686c-42ba-9846-0af94245193b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:48.792991 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 930/1831] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:48 2022] PUT /identity/v3/projects/4c250878bd204cb49f4458ace310ae9f/users/96014290ba4049ad85def453315128df/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:48.797978 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:48.808078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5289c0-511a-4e8a-9cd8-88567ceed0d0 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:48.808401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5289c0-511a-4e8a-9cd8-88567ceed0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:48.808686 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5289c0-511a-4e8a-9cd8-88567ceed0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:48.868905 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6f5289c0-511a-4e8a-9cd8-88567ceed0d0 None None] MFA Rules not processed for user `96014290ba4049ad85def453315128df`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:48.870125 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6f5289c0-511a-4e8a-9cd8-88567ceed0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:48.890614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9cef1-fe01-444b-aa2a-736b8e508ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:48.891753 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9cef1-fe01-444b-aa2a-736b8e508ff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:48.892107 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9cef1-fe01-444b-aa2a-736b8e508ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:48.892662 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce9cef1-fe01-444b-aa2a-736b8e508ff0 admin admin] PATH_INFO: `/v3/projects/af33ed30214348a3945db31b4b48fe17/users/9e3abe5f6b194beb8d7fe275d5a82cb9/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:48.937476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce9cef1-fe01-444b-aa2a-736b8e508ff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af33ed30214348a3945db31b4b48fe17, user_id=9e3abe5f6b194beb8d7fe275d5a82cb9, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:48.940904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce9cef1-fe01-444b-aa2a-736b8e508ff0 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:48.962505 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 902/1832] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:48 2022] PUT /identity/v3/projects/af33ed30214348a3945db31b4b48fe17/users/9e3abe5f6b194beb8d7fe275d5a82cb9/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:30:48.964214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9cef1-fe01-444b-aa2a-736b8e508ff0 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:48.966428 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6ce9cef1-fe01-444b-aa2a-736b8e508ff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:49.005848 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 931/1833] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:49.007325 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5289c0-511a-4e8a-9cd8-88567ceed0d0 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:49.018229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5289c0-511a-4e8a-9cd8-88567ceed0d0 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:49.049173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce9cef1-fe01-444b-aa2a-736b8e508ff0 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:49.054763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9785a-eafe-4d43-bad2-88e2abcac0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:49.055409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9785a-eafe-4d43-bad2-88e2abcac0a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:49.055691 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9785a-eafe-4d43-bad2-88e2abcac0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:49.055976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9785a-eafe-4d43-bad2-88e2abcac0a4 admin admin] PATH_INFO: `/v3/projects/0c794f4a8a9248b0834dcc49d32b7c22` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:49.057710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9785a-eafe-4d43-bad2-88e2abcac0a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c794f4a8a9248b0834dcc49d32b7c22)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:49.059568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9785a-eafe-4d43-bad2-88e2abcac0a4 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:49.084371 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3d077-68d8-4074-888b-24cbcab05294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:49.085010 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3d077-68d8-4074-888b-24cbcab05294 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:49.085290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3d077-68d8-4074-888b-24cbcab05294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:49.085569 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3d077-68d8-4074-888b-24cbcab05294 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:49.086245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3d077-68d8-4074-888b-24cbcab05294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:49.087856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3d077-68d8-4074-888b-24cbcab05294 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:49.120893 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 932/1834] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:49.121828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3d077-68d8-4074-888b-24cbcab05294 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:49.127915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-28b9785a-eafe-4d43-bad2-88e2abcac0a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c794f4a8a9248b0834dcc49d32b7c22', 'request_id': 'req-28b9785a-eafe-4d43-bad2-88e2abcac0a4'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:49.132680 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3d077-68d8-4074-888b-24cbcab05294 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:49.135694 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-28b9785a-eafe-4d43-bad2-88e2abcac0a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c794f4a8a9248b0834dcc49d32b7c22', 'request_id': 'req-28b9785a-eafe-4d43-bad2-88e2abcac0a4'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:49.146958 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-28b9785a-eafe-4d43-bad2-88e2abcac0a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c794f4a8a9248b0834dcc49d32b7c22', 'request_id': 'req-28b9785a-eafe-4d43-bad2-88e2abcac0a4'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:49.160550 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-28b9785a-eafe-4d43-bad2-88e2abcac0a4 admin admin] The token cache is being invalidate because project 0c794f4a8a9248b0834dcc49d32b7c22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:30:49.160838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-28b9785a-eafe-4d43-bad2-88e2abcac0a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:49.161944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-28b9785a-eafe-4d43-bad2-88e2abcac0a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:49.163408 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 903/1835] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:30:48 2022] DELETE /identity/v3/projects/0c794f4a8a9248b0834dcc49d32b7c22 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:49.163931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9785a-eafe-4d43-bad2-88e2abcac0a4 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:49.168399 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-28b9785a-eafe-4d43-bad2-88e2abcac0a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:49.197771 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:49.198384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:49.206754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b93af6-9467-4e5d-bc59-019a86a23c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:49.208730 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b93af6-9467-4e5d-bc59-019a86a23c71 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:49.209261 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b93af6-9467-4e5d-bc59-019a86a23c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:49.209621 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b93af6-9467-4e5d-bc59-019a86a23c71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:49.210368 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b93af6-9467-4e5d-bc59-019a86a23c71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:49.212543 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b93af6-9467-4e5d-bc59-019a86a23c71 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:49.243630 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 933/1836] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:49 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:49.245160 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbeaec-58e8-486f-84b2-ae195f7cd0b6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:49.245445 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbeaec-58e8-486f-84b2-ae195f7cd0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:49.245720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbeaec-58e8-486f-84b2-ae195f7cd0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:49.249812 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9785a-eafe-4d43-bad2-88e2abcac0a4 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:49.266711 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b6cbeaec-58e8-486f-84b2-ae195f7cd0b6 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:49.267921 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b6cbeaec-58e8-486f-84b2-ae195f7cd0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:49.311470 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306fa16-1448-4784-ab90-4da87e350a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:49.312057 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306fa16-1448-4784-ab90-4da87e350a1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:49.312365 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306fa16-1448-4784-ab90-4da87e350a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:49.312643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306fa16-1448-4784-ab90-4da87e350a1a admin admin] PATH_INFO: `/v3/projects/af33ed30214348a3945db31b4b48fe17/users/9e3abe5f6b194beb8d7fe275d5a82cb9/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:49.314578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306fa16-1448-4784-ab90-4da87e350a1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=af33ed30214348a3945db31b4b48fe17, user_id=9e3abe5f6b194beb8d7fe275d5a82cb9, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:49.317862 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306fa16-1448-4784-ab90-4da87e350a1a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:49.328973 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 904/1837] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:49 2022] PUT /identity/v3/projects/af33ed30214348a3945db31b4b48fe17/users/9e3abe5f6b194beb8d7fe275d5a82cb9/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:49.329943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306fa16-1448-4784-ab90-4da87e350a1a admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:49.331546 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2306fa16-1448-4784-ab90-4da87e350a1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:49.332446 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 934/1838] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:49.340338 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c819f-c625-4b67-84fb-8815303fc87e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:49.340640 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c819f-c625-4b67-84fb-8815303fc87e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:49.340925 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c819f-c625-4b67-84fb-8815303fc87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:49.367267 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c73c819f-c625-4b67-84fb-8815303fc87e None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:49.369047 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c73c819f-c625-4b67-84fb-8815303fc87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:49.411410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306fa16-1448-4784-ab90-4da87e350a1a admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:49.416780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d07232-1e0b-41af-8c01-fe741e864bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:49.417794 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d07232-1e0b-41af-8c01-fe741e864bdb admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:49.417910 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d07232-1e0b-41af-8c01-fe741e864bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:49.418177 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d07232-1e0b-41af-8c01-fe741e864bdb admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:49.418849 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d07232-1e0b-41af-8c01-fe741e864bdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:49.422792 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d07232-1e0b-41af-8c01-fe741e864bdb admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:49.444427 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 935/1839] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:30:49.451082 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c819f-c625-4b67-84fb-8815303fc87e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:49.463794 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 905/1840] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:49 2022] POST /identity/v3/users => generated 295 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:49.468205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d07232-1e0b-41af-8c01-fe741e864bdb admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:49.472121 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c819f-c625-4b67-84fb-8815303fc87e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:49.479091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d07232-1e0b-41af-8c01-fe741e864bdb admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:49.479689 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902ead0-faa5-4b51-a493-7a67e70743c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:49.480518 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902ead0-faa5-4b51-a493-7a67e70743c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:49.481929 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902ead0-faa5-4b51-a493-7a67e70743c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:49.482378 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902ead0-faa5-4b51-a493-7a67e70743c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:49.483321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902ead0-faa5-4b51-a493-7a67e70743c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:30:49.483639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902ead0-faa5-4b51-a493-7a67e70743c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:49.485784 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902ead0-faa5-4b51-a493-7a67e70743c6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:49.485915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098607ea-47f7-424e-80fa-e7d5308ec27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:49.486644 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098607ea-47f7-424e-80fa-e7d5308ec27e admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:49.487024 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098607ea-47f7-424e-80fa-e7d5308ec27e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:49.487394 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098607ea-47f7-424e-80fa-e7d5308ec27e admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:49.488988 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098607ea-47f7-424e-80fa-e7d5308ec27e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:49.491716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098607ea-47f7-424e-80fa-e7d5308ec27e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:49.504364 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 906/1841] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:49.508240 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 936/1842] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:30:49.511704 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098607ea-47f7-424e-80fa-e7d5308ec27e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:49.513116 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902ead0-faa5-4b51-a493-7a67e70743c6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:49.524353 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098607ea-47f7-424e-80fa-e7d5308ec27e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:49.531529 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a22596-11c5-4339-8ea4-b46a5264bf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:49.532378 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a22596-11c5-4339-8ea4-b46a5264bf5c admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:49.532659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a22596-11c5-4339-8ea4-b46a5264bf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:49.532986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a22596-11c5-4339-8ea4-b46a5264bf5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:49.533706 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902ead0-faa5-4b51-a493-7a67e70743c6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:49.533819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a22596-11c5-4339-8ea4-b46a5264bf5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:49.536481 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a22596-11c5-4339-8ea4-b46a5264bf5c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:49.542703 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d27c50-772b-4467-8c19-f868624fd04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:49.545149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d27c50-772b-4467-8c19-f868624fd04f admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:49.545653 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d27c50-772b-4467-8c19-f868624fd04f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:49.547236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d27c50-772b-4467-8c19-f868624fd04f admin admin] PATH_INFO: `/v3/projects/1e4506bcf73d4f4089467c2cc32cd801/users/11a94f917f3445b4a1b19384b2fb1e5b/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:49.578007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d27c50-772b-4467-8c19-f868624fd04f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e4506bcf73d4f4089467c2cc32cd801, user_id=11a94f917f3445b4a1b19384b2fb1e5b, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:49.581547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 907/1843] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:49 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:49.583110 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a22596-11c5-4339-8ea4-b46a5264bf5c admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:49.589619 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d27c50-772b-4467-8c19-f868624fd04f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:49.609723 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:49.612203 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 937/1844] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:49 2022] PUT /identity/v3/projects/1e4506bcf73d4f4089467c2cc32cd801/users/11a94f917f3445b4a1b19384b2fb1e5b/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:49.620036 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d27c50-772b-4467-8c19-f868624fd04f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:49.667406 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a22596-11c5-4339-8ea4-b46a5264bf5c admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:49.747356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd73e2-0497-41aa-9fd9-0f3566bb4e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:49.747994 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd73e2-0497-41aa-9fd9-0f3566bb4e1b admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:49.748449 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd73e2-0497-41aa-9fd9-0f3566bb4e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:49.748727 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd73e2-0497-41aa-9fd9-0f3566bb4e1b admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:49.749402 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd73e2-0497-41aa-9fd9-0f3566bb4e1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:49.751497 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd73e2-0497-41aa-9fd9-0f3566bb4e1b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:49.761734 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:49.784555 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d27c50-772b-4467-8c19-f868624fd04f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:49.789519 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 908/1845] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:49 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 12:30:49.790636 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928546d8-f883-4c88-be95-9952a7e97fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:49.791532 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928546d8-f883-4c88-be95-9952a7e97fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:49.791906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928546d8-f883-4c88-be95-9952a7e97fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:49.793132 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928546d8-f883-4c88-be95-9952a7e97fa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:49.793132 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928546d8-f883-4c88-be95-9952a7e97fa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:49.793758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd73e2-0497-41aa-9fd9-0f3566bb4e1b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:49.794992 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928546d8-f883-4c88-be95-9952a7e97fa1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:49.806638 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd73e2-0497-41aa-9fd9-0f3566bb4e1b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:49.808539 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 938/1846] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:49.812356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc2336-4874-4af2-8498-ecd30c5a20fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:49.813190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc2336-4874-4af2-8498-ecd30c5a20fa admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:49.813309 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928546d8-f883-4c88-be95-9952a7e97fa1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:49.813402 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc2336-4874-4af2-8498-ecd30c5a20fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:49.813681 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc2336-4874-4af2-8498-ecd30c5a20fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:49.814791 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc2336-4874-4af2-8498-ecd30c5a20fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:49.817060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc2336-4874-4af2-8498-ecd30c5a20fa admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:49.828282 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928546d8-f883-4c88-be95-9952a7e97fa1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:49.829879 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 909/1847] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:49.833495 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3929c-7078-45aa-b662-61e32bfcfebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:49.833901 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc2336-4874-4af2-8498-ecd30c5a20fa admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:49.834313 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3929c-7078-45aa-b662-61e32bfcfebb admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:49.834527 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3929c-7078-45aa-b662-61e32bfcfebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:49.834813 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3929c-7078-45aa-b662-61e32bfcfebb admin admin] PATH_INFO: `/v3/projects/1e4506bcf73d4f4089467c2cc32cd801/users/11a94f917f3445b4a1b19384b2fb1e5b/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:49.837537 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3929c-7078-45aa-b662-61e32bfcfebb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e4506bcf73d4f4089467c2cc32cd801, user_id=11a94f917f3445b4a1b19384b2fb1e5b, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:49.840919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef3929c-7078-45aa-b662-61e32bfcfebb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:49.850184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc2336-4874-4af2-8498-ecd30c5a20fa admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:49.864942 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 939/1848] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:49 2022] PUT /identity/v3/projects/1e4506bcf73d4f4089467c2cc32cd801/users/11a94f917f3445b4a1b19384b2fb1e5b/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:49.868262 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea66d16-9aca-45b6-a908-3a8700ed4f28 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:49.868911 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea66d16-9aca-45b6-a908-3a8700ed4f28 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:49.870880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea66d16-9aca-45b6-a908-3a8700ed4f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:49.922641 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-cea66d16-9aca-45b6-a908-3a8700ed4f28 None None] MFA Rules not processed for user `11a94f917f3445b4a1b19384b2fb1e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:49.924102 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cea66d16-9aca-45b6-a908-3a8700ed4f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:49.944169 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5aefe0-b6e7-4fd7-9863-cd71dffe0230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:49.944860 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5aefe0-b6e7-4fd7-9863-cd71dffe0230 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:49.945117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5aefe0-b6e7-4fd7-9863-cd71dffe0230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:49.945392 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5aefe0-b6e7-4fd7-9863-cd71dffe0230 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:49.946091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5aefe0-b6e7-4fd7-9863-cd71dffe0230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:49.947732 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5aefe0-b6e7-4fd7-9863-cd71dffe0230 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:49.962002 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 910/1849] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:49.963763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5aefe0-b6e7-4fd7-9863-cd71dffe0230 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:49.978645 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5aefe0-b6e7-4fd7-9863-cd71dffe0230 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:49.985573 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65eb25-c749-4890-973d-aa7d3e164763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:49.986596 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65eb25-c749-4890-973d-aa7d3e164763 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:49.986929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65eb25-c749-4890-973d-aa7d3e164763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:49.987597 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65eb25-c749-4890-973d-aa7d3e164763 admin admin] PATH_INFO: `/v3/projects/7d73f05bed43411a9cfea8785877f5d0/users/8eed8024ff49445ea6750bd9afdb4d95/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:50.009085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65eb25-c749-4890-973d-aa7d3e164763 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d73f05bed43411a9cfea8785877f5d0, user_id=8eed8024ff49445ea6750bd9afdb4d95, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:50.012961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65eb25-c749-4890-973d-aa7d3e164763 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:50.029163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 940/1850] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:50.040780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 911/1851] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:49 2022] PUT /identity/v3/projects/7d73f05bed43411a9cfea8785877f5d0/users/8eed8024ff49445ea6750bd9afdb4d95/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:30:50.043148 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea66d16-9aca-45b6-a908-3a8700ed4f28 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:50.044837 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cea66d16-9aca-45b6-a908-3a8700ed4f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:50.049247 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67c8b2-a551-4d7d-adfa-5059f0035083 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:50.049532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67c8b2-a551-4d7d-adfa-5059f0035083 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:50.049936 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67c8b2-a551-4d7d-adfa-5059f0035083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:50.086212 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-9c67c8b2-a551-4d7d-adfa-5059f0035083 None None] MFA Rules not processed for user `8eed8024ff49445ea6750bd9afdb4d95`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:50.087492 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9c67c8b2-a551-4d7d-adfa-5059f0035083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:50.094521 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:50.196539 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 912/1852] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:30:50.197549 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67c8b2-a551-4d7d-adfa-5059f0035083 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:50.246515 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea66d16-9aca-45b6-a908-3a8700ed4f28 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:50.271580 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67c8b2-a551-4d7d-adfa-5059f0035083 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:50.277630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9cfbc4-0d3d-457e-ba2e-cd72fbc2c7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:50.279345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cfbc4-0d3d-457e-ba2e-cd72fbc2c7ab admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:50.279627 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cfbc4-0d3d-457e-ba2e-cd72fbc2c7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:50.280574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cfbc4-0d3d-457e-ba2e-cd72fbc2c7ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:50.281660 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9cfbc4-0d3d-457e-ba2e-cd72fbc2c7ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:50.284292 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9cfbc4-0d3d-457e-ba2e-cd72fbc2c7ab admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:50.339552 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 913/1853] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:50 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:50.340681 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9cfbc4-0d3d-457e-ba2e-cd72fbc2c7ab admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:50.372045 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5d94f-b978-418c-9d82-e8d49a3a6935 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:50.372718 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5d94f-b978-418c-9d82-e8d49a3a6935 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:50.373001 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5d94f-b978-418c-9d82-e8d49a3a6935 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:50.373437 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5d94f-b978-418c-9d82-e8d49a3a6935 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:50.375925 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c4b5d94f-b978-418c-9d82-e8d49a3a6935 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:50.430554 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:50.453389 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9cfbc4-0d3d-457e-ba2e-cd72fbc2c7ab admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:50.459728 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30f6ac-c834-473c-bbee-529f0832d322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:50.460969 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30f6ac-c834-473c-bbee-529f0832d322 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:50.462070 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30f6ac-c834-473c-bbee-529f0832d322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:50.462428 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30f6ac-c834-473c-bbee-529f0832d322 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:50.463180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30f6ac-c834-473c-bbee-529f0832d322 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:50.468002 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30f6ac-c834-473c-bbee-529f0832d322 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:50.501008 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 914/1854] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:50 2022] POST /identity/v3/users => generated 299 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:50.504904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30f6ac-c834-473c-bbee-529f0832d322 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:50.521910 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5d94f-b978-418c-9d82-e8d49a3a6935 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:50.524808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30f6ac-c834-473c-bbee-529f0832d322 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:50.526044 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5d94f-b978-418c-9d82-e8d49a3a6935 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:50.529761 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0565b-d487-4f30-bcd3-2ac031bbcc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:50.530383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0565b-d487-4f30-bcd3-2ac031bbcc25 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:50.530664 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0565b-d487-4f30-bcd3-2ac031bbcc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:50.530945 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0565b-d487-4f30-bcd3-2ac031bbcc25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:50.531639 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0565b-d487-4f30-bcd3-2ac031bbcc25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:50.533304 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0565b-d487-4f30-bcd3-2ac031bbcc25 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:50.549301 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 915/1855] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:50.550649 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0565b-d487-4f30-bcd3-2ac031bbcc25 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:50.553075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 941/1856] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:50.571534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0565b-d487-4f30-bcd3-2ac031bbcc25 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:50.585104 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9b40c-2484-4558-81a4-006e76772544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:50.586070 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9b40c-2484-4558-81a4-006e76772544 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:50.586458 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9b40c-2484-4558-81a4-006e76772544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:50.586810 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9b40c-2484-4558-81a4-006e76772544 admin admin] PATH_INFO: `/v3/projects/0c82238075454cd3a8d47433a343d850/users/a1ad6b430b084b66b89d026e616dbb7b/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:50.599756 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:50.608820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f9b40c-2484-4558-81a4-006e76772544 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c82238075454cd3a8d47433a343d850, user_id=a1ad6b430b084b66b89d026e616dbb7b, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:50.612709 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f9b40c-2484-4558-81a4-006e76772544 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:50.649608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 916/1857] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:50 2022] PUT /identity/v3/projects/0c82238075454cd3a8d47433a343d850/users/a1ad6b430b084b66b89d026e616dbb7b/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:50.650775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5d94f-b978-418c-9d82-e8d49a3a6935 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:50.671771 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5d94f-b978-418c-9d82-e8d49a3a6935 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:50.786218 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d107a8-e39c-4379-96de-9b42426866f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:50.786873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d107a8-e39c-4379-96de-9b42426866f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:50.787184 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d107a8-e39c-4379-96de-9b42426866f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:50.787416 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d107a8-e39c-4379-96de-9b42426866f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:50.790312 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d107a8-e39c-4379-96de-9b42426866f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:50.792106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d107a8-e39c-4379-96de-9b42426866f9 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:50.817410 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 942/1858] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:50.821658 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9b40c-2484-4558-81a4-006e76772544 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:50.838107 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9b40c-2484-4558-81a4-006e76772544 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:50.848370 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acef1eb-81b8-49ed-bfb7-f1b84094622b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:50.849069 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acef1eb-81b8-49ed-bfb7-f1b84094622b admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:50.849354 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acef1eb-81b8-49ed-bfb7-f1b84094622b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:50.850000 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acef1eb-81b8-49ed-bfb7-f1b84094622b admin admin] PATH_INFO: `/v3/projects/0c82238075454cd3a8d47433a343d850/users/a1ad6b430b084b66b89d026e616dbb7b/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:50.857637 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acef1eb-81b8-49ed-bfb7-f1b84094622b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c82238075454cd3a8d47433a343d850, user_id=a1ad6b430b084b66b89d026e616dbb7b, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:50.860549 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acef1eb-81b8-49ed-bfb7-f1b84094622b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:50.886530 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 917/1859] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:50 2022] PUT /identity/v3/projects/0c82238075454cd3a8d47433a343d850/users/a1ad6b430b084b66b89d026e616dbb7b/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:50.897989 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836aacb-2d5e-4689-879f-a8d81ba7b9bb None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:50.898361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836aacb-2d5e-4689-879f-a8d81ba7b9bb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:50.898614 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836aacb-2d5e-4689-879f-a8d81ba7b9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:50.962598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9836aacb-2d5e-4689-879f-a8d81ba7b9bb None None] MFA Rules not processed for user `a1ad6b430b084b66b89d026e616dbb7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:50.965958 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9836aacb-2d5e-4689-879f-a8d81ba7b9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:51.052052 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb30ec-5b48-4a74-9d27-f906d9ebd7cd None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:51.052371 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb30ec-5b48-4a74-9d27-f906d9ebd7cd None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:51.052650 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb30ec-5b48-4a74-9d27-f906d9ebd7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:51.120824 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-b1cb30ec-5b48-4a74-9d27-f906d9ebd7cd None None] MFA Rules not processed for user `9e3abe5f6b194beb8d7fe275d5a82cb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:51.122862 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b1cb30ec-5b48-4a74-9d27-f906d9ebd7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:51.128458 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 943/1860] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:50 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:51.145242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836aacb-2d5e-4689-879f-a8d81ba7b9bb None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:51.228612 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:51.242653 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836aacb-2d5e-4689-879f-a8d81ba7b9bb None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:51.247852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 918/1861] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:30:51.261154 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc9038-09e5-405e-8354-49e03a522c42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:51.261778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc9038-09e5-405e-8354-49e03a522c42 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:51.262070 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc9038-09e5-405e-8354-49e03a522c42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:51.262345 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc9038-09e5-405e-8354-49e03a522c42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:51.278574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dc9038-09e5-405e-8354-49e03a522c42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:51.280787 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dc9038-09e5-405e-8354-49e03a522c42 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:51.294445 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 944/1862] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:51.391753 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb30ec-5b48-4a74-9d27-f906d9ebd7cd None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:51.463510 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb30ec-5b48-4a74-9d27-f906d9ebd7cd None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:51.468952 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43374e36-c0ad-48dc-a5b2-d5a2dc6f4bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:51.469653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43374e36-c0ad-48dc-a5b2-d5a2dc6f4bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:51.469937 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43374e36-c0ad-48dc-a5b2-d5a2dc6f4bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:51.470213 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43374e36-c0ad-48dc-a5b2-d5a2dc6f4bfd admin admin] PATH_INFO: `/v3/users/8eed8024ff49445ea6750bd9afdb4d95` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:51.471402 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43374e36-c0ad-48dc-a5b2-d5a2dc6f4bfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eed8024ff49445ea6750bd9afdb4d95)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:51.473937 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43374e36-c0ad-48dc-a5b2-d5a2dc6f4bfd admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:51.540644 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-43374e36-c0ad-48dc-a5b2-d5a2dc6f4bfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eed8024ff49445ea6750bd9afdb4d95'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:51.548210 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-43374e36-c0ad-48dc-a5b2-d5a2dc6f4bfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eed8024ff49445ea6750bd9afdb4d95'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:51.553112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:51.564559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-43374e36-c0ad-48dc-a5b2-d5a2dc6f4bfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eed8024ff49445ea6750bd9afdb4d95'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:51.565360 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-43374e36-c0ad-48dc-a5b2-d5a2dc6f4bfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eed8024ff49445ea6750bd9afdb4d95'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:51.566260 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-43374e36-c0ad-48dc-a5b2-d5a2dc6f4bfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eed8024ff49445ea6750bd9afdb4d95'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:51.589584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 919/1863] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:30:51 2022] DELETE /identity/v3/users/8eed8024ff49445ea6750bd9afdb4d95 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:51.593899 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5bb18-c5b4-4aae-acd6-7ee3a00ba26e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:51.594287 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5bb18-c5b4-4aae-acd6-7ee3a00ba26e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:51.594716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5bb18-c5b4-4aae-acd6-7ee3a00ba26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:51.623189 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-13b5bb18-c5b4-4aae-acd6-7ee3a00ba26e None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:51.624761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-13b5bb18-c5b4-4aae-acd6-7ee3a00ba26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:51.714021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 945/1864] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:30:51.726336 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43374e36-c0ad-48dc-a5b2-d5a2dc6f4bfd admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:51.727845 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-43374e36-c0ad-48dc-a5b2-d5a2dc6f4bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:51.816342 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43374e36-c0ad-48dc-a5b2-d5a2dc6f4bfd admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:51.820596 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0f2bd-d9db-4f58-bfba-c3b753e12502 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:51.821350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0f2bd-d9db-4f58-bfba-c3b753e12502 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:51.821764 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0f2bd-d9db-4f58-bfba-c3b753e12502 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:51.822118 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0f2bd-d9db-4f58-bfba-c3b753e12502 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:51.834550 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb0f2bd-d9db-4f58-bfba-c3b753e12502 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:51.836148 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb0f2bd-d9db-4f58-bfba-c3b753e12502 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:51.847161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 920/1865] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:52.062874 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5bb18-c5b4-4aae-acd6-7ee3a00ba26e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:52.064624 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-13b5bb18-c5b4-4aae-acd6-7ee3a00ba26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:52.079822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5bb18-c5b4-4aae-acd6-7ee3a00ba26e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:52.085867 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81dfd7d-94da-422e-9563-bbe8cd56db1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:52.087168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81dfd7d-94da-422e-9563-bbe8cd56db1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:52.087646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81dfd7d-94da-422e-9563-bbe8cd56db1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:52.088167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81dfd7d-94da-422e-9563-bbe8cd56db1c admin admin] PATH_INFO: `/v3/users/23a86fdf37434ee9af60be66d66b22af` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:52.089853 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81dfd7d-94da-422e-9563-bbe8cd56db1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=23a86fdf37434ee9af60be66d66b22af)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:52.092420 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81dfd7d-94da-422e-9563-bbe8cd56db1c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:52.159537 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e81dfd7d-94da-422e-9563-bbe8cd56db1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23a86fdf37434ee9af60be66d66b22af'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:52.178375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e81dfd7d-94da-422e-9563-bbe8cd56db1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23a86fdf37434ee9af60be66d66b22af'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:52.179315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e81dfd7d-94da-422e-9563-bbe8cd56db1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23a86fdf37434ee9af60be66d66b22af'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:52.179924 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e81dfd7d-94da-422e-9563-bbe8cd56db1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23a86fdf37434ee9af60be66d66b22af'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:52.192943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0f2bd-d9db-4f58-bfba-c3b753e12502 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:52.194452 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5cb0f2bd-d9db-4f58-bfba-c3b753e12502 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:52.200897 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e81dfd7d-94da-422e-9563-bbe8cd56db1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23a86fdf37434ee9af60be66d66b22af'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:52.215456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 946/1866] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:30:52 2022] DELETE /identity/v3/users/23a86fdf37434ee9af60be66d66b22af => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:30:52.222046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932bcf1-282e-4403-b3ed-8ae982077c97 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:52.222861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932bcf1-282e-4403-b3ed-8ae982077c97 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:52.223140 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932bcf1-282e-4403-b3ed-8ae982077c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:52.226412 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0f2bd-d9db-4f58-bfba-c3b753e12502 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:52.263534 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d932bcf1-282e-4403-b3ed-8ae982077c97 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:52.279926 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d932bcf1-282e-4403-b3ed-8ae982077c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:52.312756 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a424f87-2619-4291-9ac1-3fdebb483a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:52.314212 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a424f87-2619-4291-9ac1-3fdebb483a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:52.314868 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a424f87-2619-4291-9ac1-3fdebb483a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:52.315143 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a424f87-2619-4291-9ac1-3fdebb483a05 admin admin] PATH_INFO: `/v3/projects/7d73f05bed43411a9cfea8785877f5d0` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:52.318137 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a424f87-2619-4291-9ac1-3fdebb483a05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d73f05bed43411a9cfea8785877f5d0)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:52.320560 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a424f87-2619-4291-9ac1-3fdebb483a05 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:52.393748 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 947/1867] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:52.396528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1bb1d-f29f-4dd6-9fce-bcbfeb2e6a85 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:52.397533 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1bb1d-f29f-4dd6-9fce-bcbfeb2e6a85 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:52.397918 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1bb1d-f29f-4dd6-9fce-bcbfeb2e6a85 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:52.400686 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 948/1868] 10.222.0.20 () {54 vars in 921 bytes} [Sat Aug 6 12:30:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:30:52.427728 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7a424f87-2619-4291-9ac1-3fdebb483a05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d73f05bed43411a9cfea8785877f5d0', 'request_id': 'req-7a424f87-2619-4291-9ac1-3fdebb483a05'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:52.428321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1bb1d-f29f-4dd6-9fce-bcbfeb2e6a85 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:52.429005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7a424f87-2619-4291-9ac1-3fdebb483a05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d73f05bed43411a9cfea8785877f5d0', 'request_id': 'req-7a424f87-2619-4291-9ac1-3fdebb483a05'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:52.432973 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-41d1bb1d-f29f-4dd6-9fce-bcbfeb2e6a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:52.444143 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7a424f87-2619-4291-9ac1-3fdebb483a05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d73f05bed43411a9cfea8785877f5d0', 'request_id': 'req-7a424f87-2619-4291-9ac1-3fdebb483a05'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:52.444285 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:52.466016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7a424f87-2619-4291-9ac1-3fdebb483a05 admin admin] The token cache is being invalidate because project 7d73f05bed43411a9cfea8785877f5d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:30:52.466016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7a424f87-2619-4291-9ac1-3fdebb483a05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:52.466016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7a424f87-2619-4291-9ac1-3fdebb483a05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:52.466016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 921/1869] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:30:52 2022] DELETE /identity/v3/projects/7d73f05bed43411a9cfea8785877f5d0 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:52.466016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bc52a-3dcd-43dc-80dd-a3723f499cca None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:52.466337 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bc52a-3dcd-43dc-80dd-a3723f499cca None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:52.466337 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bc52a-3dcd-43dc-80dd-a3723f499cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:52.489423 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-599bc52a-3dcd-43dc-80dd-a3723f499cca None None] MFA Rules not processed for user `e2244655ba304b9c980b5e68772eefc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:52.491071 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-599bc52a-3dcd-43dc-80dd-a3723f499cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:52.505792 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:52.551362 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1bb1d-f29f-4dd6-9fce-bcbfeb2e6a85 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:52.555335 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e58777-0940-4d40-8e9b-d04c9367b233 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:52.556253 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e58777-0940-4d40-8e9b-d04c9367b233 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:52.556795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e58777-0940-4d40-8e9b-d04c9367b233 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:52.557263 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e58777-0940-4d40-8e9b-d04c9367b233 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:52.559250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-31e58777-0940-4d40-8e9b-d04c9367b233 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:52.613657 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 922/1870] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Aug 6 12:30:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:52.614531 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599bc52a-3dcd-43dc-80dd-a3723f499cca None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:52.616397 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-599bc52a-3dcd-43dc-80dd-a3723f499cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:52.649742 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e58777-0940-4d40-8e9b-d04c9367b233 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:52.651699 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e58777-0940-4d40-8e9b-d04c9367b233 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:52.665673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 949/1871] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:52.666543 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e58777-0940-4d40-8e9b-d04c9367b233 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:52.668498 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-31e58777-0940-4d40-8e9b-d04c9367b233 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:52.683703 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e58777-0940-4d40-8e9b-d04c9367b233 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:52.685665 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599bc52a-3dcd-43dc-80dd-a3723f499cca None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:52.690428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa51343-30e4-43b2-8cc2-ee8d1345432b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:52.691680 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa51343-30e4-43b2-8cc2-ee8d1345432b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:52.691959 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa51343-30e4-43b2-8cc2-ee8d1345432b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:52.692400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa51343-30e4-43b2-8cc2-ee8d1345432b admin admin] PATH_INFO: `/v3/users/a1ad6b430b084b66b89d026e616dbb7b` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:52.693714 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa51343-30e4-43b2-8cc2-ee8d1345432b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1ad6b430b084b66b89d026e616dbb7b)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:52.695638 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa51343-30e4-43b2-8cc2-ee8d1345432b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:52.695928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8192c-250e-469d-bb0b-62514cf69969 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:52.696600 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8192c-250e-469d-bb0b-62514cf69969 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:52.698471 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8192c-250e-469d-bb0b-62514cf69969 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:52.699535 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8192c-250e-469d-bb0b-62514cf69969 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:52.702653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6cc8192c-250e-469d-bb0b-62514cf69969 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:52.777849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1aa51343-30e4-43b2-8cc2-ee8d1345432b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1ad6b430b084b66b89d026e616dbb7b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:52.800860 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1aa51343-30e4-43b2-8cc2-ee8d1345432b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1ad6b430b084b66b89d026e616dbb7b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:52.801814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1aa51343-30e4-43b2-8cc2-ee8d1345432b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1ad6b430b084b66b89d026e616dbb7b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:52.802459 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1aa51343-30e4-43b2-8cc2-ee8d1345432b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1ad6b430b084b66b89d026e616dbb7b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:52.809717 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8192c-250e-469d-bb0b-62514cf69969 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:52.813096 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8192c-250e-469d-bb0b-62514cf69969 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:52.818487 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1aa51343-30e4-43b2-8cc2-ee8d1345432b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1ad6b430b084b66b89d026e616dbb7b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:52.829968 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 923/1872] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:52.830753 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8192c-250e-469d-bb0b-62514cf69969 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:52.832565 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6cc8192c-250e-469d-bb0b-62514cf69969 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:52.850530 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8192c-250e-469d-bb0b-62514cf69969 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:52.851049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 950/1873] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:30:52 2022] DELETE /identity/v3/users/a1ad6b430b084b66b89d026e616dbb7b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:52.933002 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:52.952634 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3785f3e-956d-491d-8409-c0c6a046ecef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2244655ba304b9c980b5e68772eefc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:52.955038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3785f3e-956d-491d-8409-c0c6a046ecef service placement] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:52.955663 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3785f3e-956d-491d-8409-c0c6a046ecef service placement] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:52.956247 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3785f3e-956d-491d-8409-c0c6a046ecef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:52.962411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f3785f3e-956d-491d-8409-c0c6a046ecef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:52.983309 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3785f3e-956d-491d-8409-c0c6a046ecef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:52.985785 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3785f3e-956d-491d-8409-c0c6a046ecef service placement] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:53.014559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 924/1874] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:53.228918 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa51343-30e4-43b2-8cc2-ee8d1345432b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:53.231097 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1aa51343-30e4-43b2-8cc2-ee8d1345432b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:53.277926 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:53.330311 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa51343-30e4-43b2-8cc2-ee8d1345432b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:53.337246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353387a-cf7f-4f56-83aa-2a03df36c795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:53.337944 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353387a-cf7f-4f56-83aa-2a03df36c795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:53.338227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353387a-cf7f-4f56-83aa-2a03df36c795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:53.338498 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353387a-cf7f-4f56-83aa-2a03df36c795 admin admin] PATH_INFO: `/v3/projects/0c82238075454cd3a8d47433a343d850` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:53.339731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353387a-cf7f-4f56-83aa-2a03df36c795 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c82238075454cd3a8d47433a343d850)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:53.341798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353387a-cf7f-4f56-83aa-2a03df36c795 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:53.403882 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafa5ae-1d46-4b4d-a95f-f1325c15abcc None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:53.404167 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafa5ae-1d46-4b4d-a95f-f1325c15abcc None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:53.404499 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafa5ae-1d46-4b4d-a95f-f1325c15abcc None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:53.406351 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 925/1875] 10.222.0.20 () {54 vars in 921 bytes} [Sat Aug 6 12:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:30:53.411270 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f353387a-cf7f-4f56-83aa-2a03df36c795 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c82238075454cd3a8d47433a343d850', 'request_id': 'req-f353387a-cf7f-4f56-83aa-2a03df36c795'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:53.417374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f353387a-cf7f-4f56-83aa-2a03df36c795 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c82238075454cd3a8d47433a343d850', 'request_id': 'req-f353387a-cf7f-4f56-83aa-2a03df36c795'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:53.428346 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f353387a-cf7f-4f56-83aa-2a03df36c795 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c82238075454cd3a8d47433a343d850', 'request_id': 'req-f353387a-cf7f-4f56-83aa-2a03df36c795'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:53.440411 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f353387a-cf7f-4f56-83aa-2a03df36c795 admin admin] The token cache is being invalidate because project 0c82238075454cd3a8d47433a343d850 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:30:53.440878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f353387a-cf7f-4f56-83aa-2a03df36c795 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:53.442121 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f353387a-cf7f-4f56-83aa-2a03df36c795 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:53.444006 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 951/1876] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:30:53 2022] DELETE /identity/v3/projects/0c82238075454cd3a8d47433a343d850 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:53.445170 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38440580-897c-4784-883c-0c4a997457f4 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:53.445619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38440580-897c-4784-883c-0c4a997457f4 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:53.445785 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38440580-897c-4784-883c-0c4a997457f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:53.449122 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bf4ed-0fbe-4263-8879-5bc5c530d371 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:53.449122 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bf4ed-0fbe-4263-8879-5bc5c530d371 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:53.449122 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bf4ed-0fbe-4263-8879-5bc5c530d371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:53.473964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-7e3bf4ed-0fbe-4263-8879-5bc5c530d371 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:53.475759 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7e3bf4ed-0fbe-4263-8879-5bc5c530d371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:53.500106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-38440580-897c-4784-883c-0c4a997457f4 None None] MFA Rules not processed for user `e2244655ba304b9c980b5e68772eefc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:53.511831 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-38440580-897c-4784-883c-0c4a997457f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:53.586370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 952/1877] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:30:53.589852 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3bf4ed-0fbe-4263-8879-5bc5c530d371 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:53.591293 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7e3bf4ed-0fbe-4263-8879-5bc5c530d371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:53.617202 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3bf4ed-0fbe-4263-8879-5bc5c530d371 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:53.622446 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9184d-29f9-4f58-8556-3ac302ee5357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:53.623106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9184d-29f9-4f58-8556-3ac302ee5357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:53.623386 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9184d-29f9-4f58-8556-3ac302ee5357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:53.623665 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9184d-29f9-4f58-8556-3ac302ee5357 admin admin] PATH_INFO: `/v3/projects/832ae09639224ab1834e30e816f7d09f` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:53.624886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc9184d-29f9-4f58-8556-3ac302ee5357 admin admin] RBAC: Authorizing `identity:delete_project(project_id=832ae09639224ab1834e30e816f7d09f)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:53.626780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc9184d-29f9-4f58-8556-3ac302ee5357 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:53.652112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 926/1878] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Aug 6 12:30:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:30:53.654517 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38440580-897c-4784-883c-0c4a997457f4 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:53.656006 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-38440580-897c-4784-883c-0c4a997457f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:53.672343 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38440580-897c-4784-883c-0c4a997457f4 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:53.678492 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece91616-1f42-4953-9c07-982279ba4d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:53.679195 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece91616-1f42-4953-9c07-982279ba4d39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:53.679502 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece91616-1f42-4953-9c07-982279ba4d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:53.679804 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece91616-1f42-4953-9c07-982279ba4d39 admin admin] PATH_INFO: `/v3/users/1f11ddea61374b0dbf660d699cb9b121` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:53.681149 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece91616-1f42-4953-9c07-982279ba4d39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f11ddea61374b0dbf660d699cb9b121)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:53.695088 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece91616-1f42-4953-9c07-982279ba4d39 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:53.697305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1fc9184d-29f9-4f58-8556-3ac302ee5357 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '832ae09639224ab1834e30e816f7d09f', 'request_id': 'req-1fc9184d-29f9-4f58-8556-3ac302ee5357'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:53.698871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:53.705807 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1fc9184d-29f9-4f58-8556-3ac302ee5357 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '832ae09639224ab1834e30e816f7d09f', 'request_id': 'req-1fc9184d-29f9-4f58-8556-3ac302ee5357'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:53.722127 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1fc9184d-29f9-4f58-8556-3ac302ee5357 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '832ae09639224ab1834e30e816f7d09f', 'request_id': 'req-1fc9184d-29f9-4f58-8556-3ac302ee5357'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:53.740194 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1fc9184d-29f9-4f58-8556-3ac302ee5357 admin admin] The token cache is being invalidate because project 832ae09639224ab1834e30e816f7d09f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:30:53.741256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1fc9184d-29f9-4f58-8556-3ac302ee5357 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:53.742443 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1fc9184d-29f9-4f58-8556-3ac302ee5357 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:53.745960 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 953/1879] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:30:53 2022] DELETE /identity/v3/projects/832ae09639224ab1834e30e816f7d09f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:53.747454 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907b48c-5dfa-4766-836e-dbf44dc68823 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:53.747908 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907b48c-5dfa-4766-836e-dbf44dc68823 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:53.748190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907b48c-5dfa-4766-836e-dbf44dc68823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:53.778502 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1907b48c-5dfa-4766-836e-dbf44dc68823 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:53.785103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1907b48c-5dfa-4766-836e-dbf44dc68823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:53.791202 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ece91616-1f42-4953-9c07-982279ba4d39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f11ddea61374b0dbf660d699cb9b121'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:53.803153 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ece91616-1f42-4953-9c07-982279ba4d39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f11ddea61374b0dbf660d699cb9b121'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:53.824925 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ece91616-1f42-4953-9c07-982279ba4d39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f11ddea61374b0dbf660d699cb9b121'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:53.827066 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ece91616-1f42-4953-9c07-982279ba4d39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f11ddea61374b0dbf660d699cb9b121'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:53.827947 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ece91616-1f42-4953-9c07-982279ba4d39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f11ddea61374b0dbf660d699cb9b121'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:53.850307 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 927/1880] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:30:53 2022] DELETE /identity/v3/users/1f11ddea61374b0dbf660d699cb9b121 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:53.852253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece91616-1f42-4953-9c07-982279ba4d39 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:53.855016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ece91616-1f42-4953-9c07-982279ba4d39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:53.869789 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 954/1881] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:53.870803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907b48c-5dfa-4766-836e-dbf44dc68823 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:53.872166 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1907b48c-5dfa-4766-836e-dbf44dc68823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:53.954511 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece91616-1f42-4953-9c07-982279ba4d39 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:53.961036 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d3b7d-730b-4e4f-b483-a8a9d5a1a3a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:53.962302 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d3b7d-730b-4e4f-b483-a8a9d5a1a3a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:53.963162 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d3b7d-730b-4e4f-b483-a8a9d5a1a3a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:53.964015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d3b7d-730b-4e4f-b483-a8a9d5a1a3a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:53.968410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ec9d3b7d-730b-4e4f-b483-a8a9d5a1a3a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:53.971494 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907b48c-5dfa-4766-836e-dbf44dc68823 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:54.006185 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:54.066940 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d3b7d-730b-4e4f-b483-a8a9d5a1a3a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:54.068609 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d3b7d-730b-4e4f-b483-a8a9d5a1a3a7 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:54.083582 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 928/1882] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:54.084820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d3b7d-730b-4e4f-b483-a8a9d5a1a3a7 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:54.086538 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ec9d3b7d-730b-4e4f-b483-a8a9d5a1a3a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:54.110601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0592df-82b8-45ea-b286-b059fca3e65d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2244655ba304b9c980b5e68772eefc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:54.111299 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0592df-82b8-45ea-b286-b059fca3e65d service placement] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:54.111517 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0592df-82b8-45ea-b286-b059fca3e65d service placement] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:54.111798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0592df-82b8-45ea-b286-b059fca3e65d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:54.115885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0d0592df-82b8-45ea-b286-b059fca3e65d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:54.144650 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0592df-82b8-45ea-b286-b059fca3e65d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:54.146297 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0592df-82b8-45ea-b286-b059fca3e65d service placement] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:54.176924 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d3b7d-730b-4e4f-b483-a8a9d5a1a3a7 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:54.182820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00575a5-2806-47e3-af7f-1b4998c25ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:54.183568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00575a5-2806-47e3-af7f-1b4998c25ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:54.183846 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00575a5-2806-47e3-af7f-1b4998c25ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:54.184120 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00575a5-2806-47e3-af7f-1b4998c25ea4 admin admin] PATH_INFO: `/v3/users/a80b476746a34d98826c4290cb861501` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:54.185744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00575a5-2806-47e3-af7f-1b4998c25ea4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a80b476746a34d98826c4290cb861501)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:54.188165 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00575a5-2806-47e3-af7f-1b4998c25ea4 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:54.196817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 955/1883] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:54.196817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0592df-82b8-45ea-b286-b059fca3e65d service placement] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:54.216874 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0592df-82b8-45ea-b286-b059fca3e65d service placement] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:54.232733 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d68f8-fe0d-40b3-8e72-b875f5ced865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:54.234927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d68f8-fe0d-40b3-8e72-b875f5ced865 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:54.235162 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d68f8-fe0d-40b3-8e72-b875f5ced865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:54.235432 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d68f8-fe0d-40b3-8e72-b875f5ced865 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:54.237883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d68f8-fe0d-40b3-8e72-b875f5ced865 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:30:54.238870 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d68f8-fe0d-40b3-8e72-b875f5ced865 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:54.241501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d68f8-fe0d-40b3-8e72-b875f5ced865 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:54.265644 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:54.282309 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 956/1884] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:30:54.284416 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff106a4f-50b6-4378-9e52-92b014fcf4db None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:54.284539 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff106a4f-50b6-4378-9e52-92b014fcf4db None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:54.285431 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff106a4f-50b6-4378-9e52-92b014fcf4db None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:54.286811 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f00575a5-2806-47e3-af7f-1b4998c25ea4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a80b476746a34d98826c4290cb861501'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:54.290471 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 957/1885] 10.222.0.20 () {54 vars in 921 bytes} [Sat Aug 6 12:30:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:30:54.295307 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff106a4f-50b6-4378-9e52-92b014fcf4db None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:54.298835 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ff106a4f-50b6-4378-9e52-92b014fcf4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:54.302188 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f00575a5-2806-47e3-af7f-1b4998c25ea4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a80b476746a34d98826c4290cb861501'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:54.328188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff106a4f-50b6-4378-9e52-92b014fcf4db None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:54.334393 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f00575a5-2806-47e3-af7f-1b4998c25ea4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a80b476746a34d98826c4290cb861501'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:54.335162 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61f626-a55c-4890-af77-41f29f8ae7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:54.336010 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61f626-a55c-4890-af77-41f29f8ae7a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:54.336562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61f626-a55c-4890-af77-41f29f8ae7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:54.336984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61f626-a55c-4890-af77-41f29f8ae7a8 admin admin] PATH_INFO: `/v3/projects/3997142544354d13a045c4765fe1fb49` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:54.337565 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f00575a5-2806-47e3-af7f-1b4998c25ea4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a80b476746a34d98826c4290cb861501'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:54.338224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f00575a5-2806-47e3-af7f-1b4998c25ea4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a80b476746a34d98826c4290cb861501'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:54.339116 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61f626-a55c-4890-af77-41f29f8ae7a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3997142544354d13a045c4765fe1fb49)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:54.342871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61f626-a55c-4890-af77-41f29f8ae7a8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:54.360865 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 929/1886] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:30:54 2022] DELETE /identity/v3/users/a80b476746a34d98826c4290cb861501 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:54.361893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00575a5-2806-47e3-af7f-1b4998c25ea4 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:54.367895 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f00575a5-2806-47e3-af7f-1b4998c25ea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:54.424568 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6e61f626-a55c-4890-af77-41f29f8ae7a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3997142544354d13a045c4765fe1fb49', 'request_id': 'req-6e61f626-a55c-4890-af77-41f29f8ae7a8'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:54.440996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6e61f626-a55c-4890-af77-41f29f8ae7a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3997142544354d13a045c4765fe1fb49', 'request_id': 'req-6e61f626-a55c-4890-af77-41f29f8ae7a8'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:54.448532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:54.460077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6e61f626-a55c-4890-af77-41f29f8ae7a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3997142544354d13a045c4765fe1fb49', 'request_id': 'req-6e61f626-a55c-4890-af77-41f29f8ae7a8'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:54.484102 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6e61f626-a55c-4890-af77-41f29f8ae7a8 admin admin] The token cache is being invalidate because project 3997142544354d13a045c4765fe1fb49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:30:54.485809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6e61f626-a55c-4890-af77-41f29f8ae7a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:54.487274 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6e61f626-a55c-4890-af77-41f29f8ae7a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:54.489901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 958/1887] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:30:54 2022] DELETE /identity/v3/projects/3997142544354d13a045c4765fe1fb49 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:54.494692 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd239b1-a15e-434e-993f-63a8b1b666a9 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:54.495045 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd239b1-a15e-434e-993f-63a8b1b666a9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:54.495327 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd239b1-a15e-434e-993f-63a8b1b666a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:54.516938 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00575a5-2806-47e3-af7f-1b4998c25ea4 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:54.518549 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-bdd239b1-a15e-434e-993f-63a8b1b666a9 None None] MFA Rules not processed for user `524b728bf63c4ca88e0f25e531cf5056`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:54.520663 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bdd239b1-a15e-434e-993f-63a8b1b666a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:54.602154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163b12c-5134-4b1f-8956-a6724076aa08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:54.603465 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163b12c-5134-4b1f-8956-a6724076aa08 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:54.603754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163b12c-5134-4b1f-8956-a6724076aa08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:54.604037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163b12c-5134-4b1f-8956-a6724076aa08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:54.604777 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163b12c-5134-4b1f-8956-a6724076aa08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:54.607286 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163b12c-5134-4b1f-8956-a6724076aa08 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:54.625362 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 959/1888] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Aug 6 12:30:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:54.626361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd239b1-a15e-434e-993f-63a8b1b666a9 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:54.627799 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bdd239b1-a15e-434e-993f-63a8b1b666a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:54.643332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd239b1-a15e-434e-993f-63a8b1b666a9 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:54.644004 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 930/1889] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:54 2022] POST /identity/v3/projects => generated 393 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:54.645424 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163b12c-5134-4b1f-8956-a6724076aa08 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:54.647075 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d163b12c-5134-4b1f-8956-a6724076aa08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:54.718539 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41e8f8-b35a-4611-875f-c514ebdcdb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:54.719432 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41e8f8-b35a-4611-875f-c514ebdcdb3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:54.719783 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41e8f8-b35a-4611-875f-c514ebdcdb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:54.720125 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41e8f8-b35a-4611-875f-c514ebdcdb3a admin admin] PATH_INFO: `/v3/users/9e3abe5f6b194beb8d7fe275d5a82cb9` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:54.722657 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41e8f8-b35a-4611-875f-c514ebdcdb3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e3abe5f6b194beb8d7fe275d5a82cb9)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:54.725720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41e8f8-b35a-4611-875f-c514ebdcdb3a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:54.745051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163b12c-5134-4b1f-8956-a6724076aa08 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:54.747686 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:54.752952 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd9fd1-fa94-4411-a803-4357402eb245 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:54.754022 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd9fd1-fa94-4411-a803-4357402eb245 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:54.754497 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd9fd1-fa94-4411-a803-4357402eb245 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:54.754934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd9fd1-fa94-4411-a803-4357402eb245 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:54.767201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd9fd1-fa94-4411-a803-4357402eb245 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:54.769644 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd9fd1-fa94-4411-a803-4357402eb245 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:54.785033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 931/1890] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:54.787101 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd9fd1-fa94-4411-a803-4357402eb245 service nova] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:54.791369 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6d41e8f8-b35a-4611-875f-c514ebdcdb3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e3abe5f6b194beb8d7fe275d5a82cb9'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:54.795484 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-59cd9fd1-fa94-4411-a803-4357402eb245 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:54.807505 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6d41e8f8-b35a-4611-875f-c514ebdcdb3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e3abe5f6b194beb8d7fe275d5a82cb9'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:54.808136 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6d41e8f8-b35a-4611-875f-c514ebdcdb3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e3abe5f6b194beb8d7fe275d5a82cb9'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:54.808786 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6d41e8f8-b35a-4611-875f-c514ebdcdb3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e3abe5f6b194beb8d7fe275d5a82cb9'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:54.814689 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd9fd1-fa94-4411-a803-4357402eb245 service nova] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:54.820519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6d41e8f8-b35a-4611-875f-c514ebdcdb3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e3abe5f6b194beb8d7fe275d5a82cb9'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:54.823529 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56efe93-ebff-4ec1-99d1-47e413c68f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:54.824568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56efe93-ebff-4ec1-99d1-47e413c68f3a admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:54.824711 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56efe93-ebff-4ec1-99d1-47e413c68f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:54.825005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56efe93-ebff-4ec1-99d1-47e413c68f3a admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:54.826523 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56efe93-ebff-4ec1-99d1-47e413c68f3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:54.830353 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56efe93-ebff-4ec1-99d1-47e413c68f3a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:54.857430 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 960/1891] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:30:54 2022] DELETE /identity/v3/users/9e3abe5f6b194beb8d7fe275d5a82cb9 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:54.859103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41e8f8-b35a-4611-875f-c514ebdcdb3a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:54.868867 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6d41e8f8-b35a-4611-875f-c514ebdcdb3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:54.873361 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 932/1892] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:54 2022] POST /identity/v3/users => generated 297 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:54.880813 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56efe93-ebff-4ec1-99d1-47e413c68f3a admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:54.884157 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a56efe93-ebff-4ec1-99d1-47e413c68f3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:54.902412 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56efe93-ebff-4ec1-99d1-47e413c68f3a admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:54.953211 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:54.966887 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41e8f8-b35a-4611-875f-c514ebdcdb3a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:54.972885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf531f-365e-4043-8f3f-bf9dac3efe5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:54.973811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf531f-365e-4043-8f3f-bf9dac3efe5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:54.974259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf531f-365e-4043-8f3f-bf9dac3efe5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:54.974707 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf531f-365e-4043-8f3f-bf9dac3efe5d admin admin] PATH_INFO: `/v3/projects/35d430f3c7484eeda44df70babe6c4da` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:54.976257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf531f-365e-4043-8f3f-bf9dac3efe5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=35d430f3c7484eeda44df70babe6c4da)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:54.978805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf531f-365e-4043-8f3f-bf9dac3efe5d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:54.991356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58e2fc-0df4-4c8d-bafe-e145593d1bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:54.992395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58e2fc-0df4-4c8d-bafe-e145593d1bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:54.992760 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58e2fc-0df4-4c8d-bafe-e145593d1bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:54.993262 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58e2fc-0df4-4c8d-bafe-e145593d1bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:54.994074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58e2fc-0df4-4c8d-bafe-e145593d1bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:54.996117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58e2fc-0df4-4c8d-bafe-e145593d1bc3 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:55.008547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 933/1893] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 12:30:55.012124 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58e2fc-0df4-4c8d-bafe-e145593d1bc3 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:55.025548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58e2fc-0df4-4c8d-bafe-e145593d1bc3 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:55.031887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eea52f-e849-40ea-a8fe-18f8d6ca7ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:55.031887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eea52f-e849-40ea-a8fe-18f8d6ca7ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:55.032233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eea52f-e849-40ea-a8fe-18f8d6ca7ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:55.032617 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eea52f-e849-40ea-a8fe-18f8d6ca7ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:55.033548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eea52f-e849-40ea-a8fe-18f8d6ca7ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:55.037521 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eea52f-e849-40ea-a8fe-18f8d6ca7ba6 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:55.053858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 934/1894] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:55.055903 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eea52f-e849-40ea-a8fe-18f8d6ca7ba6 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:55.068549 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2acf531f-365e-4043-8f3f-bf9dac3efe5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35d430f3c7484eeda44df70babe6c4da', 'request_id': 'req-2acf531f-365e-4043-8f3f-bf9dac3efe5d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:55.073479 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eea52f-e849-40ea-a8fe-18f8d6ca7ba6 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:55.077755 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2acf531f-365e-4043-8f3f-bf9dac3efe5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35d430f3c7484eeda44df70babe6c4da', 'request_id': 'req-2acf531f-365e-4043-8f3f-bf9dac3efe5d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:55.090565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2acf531f-365e-4043-8f3f-bf9dac3efe5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35d430f3c7484eeda44df70babe6c4da', 'request_id': 'req-2acf531f-365e-4043-8f3f-bf9dac3efe5d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:55.123484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2acf531f-365e-4043-8f3f-bf9dac3efe5d admin admin] The token cache is being invalidate because project 35d430f3c7484eeda44df70babe6c4da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:30:55.123990 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2acf531f-365e-4043-8f3f-bf9dac3efe5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:55.125413 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2acf531f-365e-4043-8f3f-bf9dac3efe5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:55.128983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 961/1895] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:30:54 2022] DELETE /identity/v3/projects/35d430f3c7484eeda44df70babe6c4da => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:55.134680 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15af3e-32c0-403c-b8dd-75aedbaf3367 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:55.135380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15af3e-32c0-403c-b8dd-75aedbaf3367 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:55.135784 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15af3e-32c0-403c-b8dd-75aedbaf3367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:55.172879 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5db6d-a9fd-420f-9e9f-0bbd692cb747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:55.173174 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5db6d-a9fd-420f-9e9f-0bbd692cb747 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:55.173444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5db6d-a9fd-420f-9e9f-0bbd692cb747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:55.173720 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5db6d-a9fd-420f-9e9f-0bbd692cb747 admin admin] PATH_INFO: `/v3/projects/ac509f3bfdfa49149c4cc3f5d63d0932/users/eac21bfdfeb942dab5a049eb0f74a146/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:55.188694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5f15af3e-32c0-403c-b8dd-75aedbaf3367 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:55.188694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5f15af3e-32c0-403c-b8dd-75aedbaf3367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:55.204869 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 962/1896] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:55.212156 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef0203-0046-412a-a807-cd2dd875fa26 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:55.212495 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef0203-0046-412a-a807-cd2dd875fa26 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:55.212776 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef0203-0046-412a-a807-cd2dd875fa26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:55.220237 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5db6d-a9fd-420f-9e9f-0bbd692cb747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac509f3bfdfa49149c4cc3f5d63d0932, user_id=eac21bfdfeb942dab5a049eb0f74a146, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:55.224671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5db6d-a9fd-420f-9e9f-0bbd692cb747 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:55.254467 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 935/1897] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:55 2022] PUT /identity/v3/projects/ac509f3bfdfa49149c4cc3f5d63d0932/users/eac21bfdfeb942dab5a049eb0f74a146/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:55.257387 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c2ef0203-0046-412a-a807-cd2dd875fa26 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:55.258603 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c2ef0203-0046-412a-a807-cd2dd875fa26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:55.265897 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9359d8-6590-4e5f-8d61-7829228e0aa5 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:55.266174 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9359d8-6590-4e5f-8d61-7829228e0aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:55.266450 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9359d8-6590-4e5f-8d61-7829228e0aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:55.281139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:55.318913 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-7e9359d8-6590-4e5f-8d61-7829228e0aa5 None None] MFA Rules not processed for user `eac21bfdfeb942dab5a049eb0f74a146`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:55.320889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7e9359d8-6590-4e5f-8d61-7829228e0aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:55.359408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 963/1898] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:55.360588 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef0203-0046-412a-a807-cd2dd875fa26 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:55.362461 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c2ef0203-0046-412a-a807-cd2dd875fa26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:55.376178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef0203-0046-412a-a807-cd2dd875fa26 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:55.385481 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefe37e-0e14-4ae3-af2d-6c74364ce194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:55.386201 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefe37e-0e14-4ae3-af2d-6c74364ce194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:55.386485 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefe37e-0e14-4ae3-af2d-6c74364ce194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:55.386765 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefe37e-0e14-4ae3-af2d-6c74364ce194 admin admin] PATH_INFO: `/v3/projects/af33ed30214348a3945db31b4b48fe17` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:55.388044 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfefe37e-0e14-4ae3-af2d-6c74364ce194 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af33ed30214348a3945db31b4b48fe17)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:55.390139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfefe37e-0e14-4ae3-af2d-6c74364ce194 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:55.419003 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 936/1899] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:55.420073 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9359d8-6590-4e5f-8d61-7829228e0aa5 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:55.433042 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9359d8-6590-4e5f-8d61-7829228e0aa5 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:55.439465 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19214940-d2e1-4bda-b1dd-84bf4a84b6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:55.440097 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19214940-d2e1-4bda-b1dd-84bf4a84b6f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:55.440418 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19214940-d2e1-4bda-b1dd-84bf4a84b6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:55.440693 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19214940-d2e1-4bda-b1dd-84bf4a84b6f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:55.441454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19214940-d2e1-4bda-b1dd-84bf4a84b6f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:30:55.441889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19214940-d2e1-4bda-b1dd-84bf4a84b6f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:55.443540 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19214940-d2e1-4bda-b1dd-84bf4a84b6f8 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:55.458888 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cfefe37e-0e14-4ae3-af2d-6c74364ce194 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af33ed30214348a3945db31b4b48fe17', 'request_id': 'req-cfefe37e-0e14-4ae3-af2d-6c74364ce194'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:55.463793 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 937/1900] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:30:55.465655 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cfefe37e-0e14-4ae3-af2d-6c74364ce194 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af33ed30214348a3945db31b4b48fe17', 'request_id': 'req-cfefe37e-0e14-4ae3-af2d-6c74364ce194'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:55.468149 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19214940-d2e1-4bda-b1dd-84bf4a84b6f8 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:55.470109 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-19214940-d2e1-4bda-b1dd-84bf4a84b6f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:55.475890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cfefe37e-0e14-4ae3-af2d-6c74364ce194 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af33ed30214348a3945db31b4b48fe17', 'request_id': 'req-cfefe37e-0e14-4ae3-af2d-6c74364ce194'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:55.486272 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19214940-d2e1-4bda-b1dd-84bf4a84b6f8 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:55.493617 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cfefe37e-0e14-4ae3-af2d-6c74364ce194 admin admin] The token cache is being invalidate because project af33ed30214348a3945db31b4b48fe17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:30:55.494010 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cfefe37e-0e14-4ae3-af2d-6c74364ce194 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:55.494740 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cfefe37e-0e14-4ae3-af2d-6c74364ce194 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:55.500259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 964/1901] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:30:55 2022] DELETE /identity/v3/projects/af33ed30214348a3945db31b4b48fe17 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:55.500259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefe37e-0e14-4ae3-af2d-6c74364ce194 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:55.501032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:55.503489 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cfefe37e-0e14-4ae3-af2d-6c74364ce194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:55.579120 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6d43d-d605-4d9b-8f2c-b98bb6662bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:55.584054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6d43d-d605-4d9b-8f2c-b98bb6662bd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:55.584361 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6d43d-d605-4d9b-8f2c-b98bb6662bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:55.584967 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6d43d-d605-4d9b-8f2c-b98bb6662bd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:55.585628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6d43d-d605-4d9b-8f2c-b98bb6662bd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:55.594413 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6d43d-d605-4d9b-8f2c-b98bb6662bd4 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:55.633092 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefe37e-0e14-4ae3-af2d-6c74364ce194 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:55.638446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 938/1902] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:55 2022] POST /identity/v3/projects => generated 391 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:55.641013 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbcd14-cb2f-4e89-a27c-b2e794e1c28e None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:55.641486 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbcd14-cb2f-4e89-a27c-b2e794e1c28e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:55.641993 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbcd14-cb2f-4e89-a27c-b2e794e1c28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:55.642681 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475653d6-a4ea-412a-bd55-8d0d95e50066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:55.643180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475653d6-a4ea-412a-bd55-8d0d95e50066 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:55.643469 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475653d6-a4ea-412a-bd55-8d0d95e50066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:55.643796 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475653d6-a4ea-412a-bd55-8d0d95e50066 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:55.645165 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475653d6-a4ea-412a-bd55-8d0d95e50066 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:55.647521 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475653d6-a4ea-412a-bd55-8d0d95e50066 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:55.680933 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-f8bbcd14-cb2f-4e89-a27c-b2e794e1c28e None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:55.682187 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f8bbcd14-cb2f-4e89-a27c-b2e794e1c28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:55.703425 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 965/1903] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:55 2022] POST /identity/v3/projects => generated 379 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:55.704451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475653d6-a4ea-412a-bd55-8d0d95e50066 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:55.707522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-475653d6-a4ea-412a-bd55-8d0d95e50066 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:55.749950 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:55.794881 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 939/1904] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:55.794881 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bbcd14-cb2f-4e89-a27c-b2e794e1c28e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:55.806152 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475653d6-a4ea-412a-bd55-8d0d95e50066 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:55.806587 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bbcd14-cb2f-4e89-a27c-b2e794e1c28e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:55.811265 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc9662-6146-4dfd-b90b-b905f6555d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:55.811879 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc9662-6146-4dfd-b90b-b905f6555d80 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:55.811982 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bcb34-3074-4e2d-8855-a51f7ab0693e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:55.812152 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc9662-6146-4dfd-b90b-b905f6555d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:55.812477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc9662-6146-4dfd-b90b-b905f6555d80 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:55.812709 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bcb34-3074-4e2d-8855-a51f7ab0693e admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:55.812995 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bcb34-3074-4e2d-8855-a51f7ab0693e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:55.813147 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc9662-6146-4dfd-b90b-b905f6555d80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:55.813304 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bcb34-3074-4e2d-8855-a51f7ab0693e admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:55.813932 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588bcb34-3074-4e2d-8855-a51f7ab0693e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:55.815049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc9662-6146-4dfd-b90b-b905f6555d80 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:55.816459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588bcb34-3074-4e2d-8855-a51f7ab0693e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:55.838369 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 966/1905] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:55 2022] POST /identity/v3/users => generated 302 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:55.839640 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa557b-afb2-473d-869e-83cac3cd67b7 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:55.840058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa557b-afb2-473d-869e-83cac3cd67b7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:55.840366 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa557b-afb2-473d-869e-83cac3cd67b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:55.846351 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 940/1906] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:55 2022] POST /identity/v3/users => generated 290 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:55.847618 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bcb34-3074-4e2d-8855-a51f7ab0693e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:55.859203 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bcb34-3074-4e2d-8855-a51f7ab0693e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:55.862587 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e7aa557b-afb2-473d-869e-83cac3cd67b7 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:55.863822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e7aa557b-afb2-473d-869e-83cac3cd67b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:55.863921 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e903a-1e4b-4195-8843-eb2a47732d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:55.864526 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e903a-1e4b-4195-8843-eb2a47732d0a admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:55.864805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e903a-1e4b-4195-8843-eb2a47732d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:55.865077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e903a-1e4b-4195-8843-eb2a47732d0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:55.866031 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e903a-1e4b-4195-8843-eb2a47732d0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:55.867674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e903a-1e4b-4195-8843-eb2a47732d0a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:55.872107 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 967/1907] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:55.872918 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa557b-afb2-473d-869e-83cac3cd67b7 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:55.882098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 941/1908] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:55.882098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e903a-1e4b-4195-8843-eb2a47732d0a admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:55.885847 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa557b-afb2-473d-869e-83cac3cd67b7 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:55.890669 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e903a-1e4b-4195-8843-eb2a47732d0a admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:55.890800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39de83-96f6-4f91-ab8c-fc46ab4adaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:55.891445 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39de83-96f6-4f91-ab8c-fc46ab4adaad admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:55.891731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39de83-96f6-4f91-ab8c-fc46ab4adaad admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:55.892007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39de83-96f6-4f91-ab8c-fc46ab4adaad admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:55.892738 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39de83-96f6-4f91-ab8c-fc46ab4adaad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:55.894963 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39de83-96f6-4f91-ab8c-fc46ab4adaad admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:55.896128 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b1d580-bed3-4edf-9ae9-c9ae2d8b0aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:55.896772 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b1d580-bed3-4edf-9ae9-c9ae2d8b0aba admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:55.897045 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b1d580-bed3-4edf-9ae9-c9ae2d8b0aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:55.897435 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b1d580-bed3-4edf-9ae9-c9ae2d8b0aba admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:55.898131 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b1d580-bed3-4edf-9ae9-c9ae2d8b0aba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:30:55.898439 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b1d580-bed3-4edf-9ae9-c9ae2d8b0aba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:55.900025 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b1d580-bed3-4edf-9ae9-c9ae2d8b0aba admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:55.905165 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 968/1909] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:55.905938 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39de83-96f6-4f91-ab8c-fc46ab4adaad admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:55.915894 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 942/1910] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:30:55.915894 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b1d580-bed3-4edf-9ae9-c9ae2d8b0aba admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:55.918835 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39de83-96f6-4f91-ab8c-fc46ab4adaad admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:55.922969 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b1d580-bed3-4edf-9ae9-c9ae2d8b0aba admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:55.923648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4389981-7ddc-440f-8cf5-5d08cb6e4869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:55.924313 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4389981-7ddc-440f-8cf5-5d08cb6e4869 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:55.924673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4389981-7ddc-440f-8cf5-5d08cb6e4869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:55.924915 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4389981-7ddc-440f-8cf5-5d08cb6e4869 admin admin] PATH_INFO: `/v3/projects/05076cfafedc42a792b60a3de8775728/users/186500f6f7bb4836a2c25192a5c4102f/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:55.927511 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5b273-e391-44e0-aaca-380dcd341810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:55.928074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5b273-e391-44e0-aaca-380dcd341810 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:55.928378 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5b273-e391-44e0-aaca-380dcd341810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:55.928656 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5b273-e391-44e0-aaca-380dcd341810 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:55.929332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5b273-e391-44e0-aaca-380dcd341810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:55.930903 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5b273-e391-44e0-aaca-380dcd341810 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:55.939424 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4389981-7ddc-440f-8cf5-5d08cb6e4869 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05076cfafedc42a792b60a3de8775728, user_id=186500f6f7bb4836a2c25192a5c4102f, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:55.940525 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 943/1911] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:55.941841 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5b273-e391-44e0-aaca-380dcd341810 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:55.942843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4389981-7ddc-440f-8cf5-5d08cb6e4869 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:55.955064 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5b273-e391-44e0-aaca-380dcd341810 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:55.958739 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 969/1912] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:55 2022] PUT /identity/v3/projects/05076cfafedc42a792b60a3de8775728/users/186500f6f7bb4836a2c25192a5c4102f/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:55.959670 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4389981-7ddc-440f-8cf5-5d08cb6e4869 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:56.026228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:56.039400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4389981-7ddc-440f-8cf5-5d08cb6e4869 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:56.042883 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1317d718-b04c-40ae-9ab0-110c191f3335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:56.043500 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1317d718-b04c-40ae-9ab0-110c191f3335 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:56.043776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1317d718-b04c-40ae-9ab0-110c191f3335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:56.044048 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1317d718-b04c-40ae-9ab0-110c191f3335 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:56.045246 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1317d718-b04c-40ae-9ab0-110c191f3335 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:56.045842 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a21c53a-9fb8-4f9f-9064-7ada9510ea20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:56.046718 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1317d718-b04c-40ae-9ab0-110c191f3335 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:56.046842 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21c53a-9fb8-4f9f-9064-7ada9510ea20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:56.047178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21c53a-9fb8-4f9f-9064-7ada9510ea20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:56.047529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21c53a-9fb8-4f9f-9064-7ada9510ea20 admin admin] PATH_INFO: `/v3/projects/5ba6d5b00cc14526819421a6a3b290fc/users/df925c3f452c4a7d99ba942e20607b20/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:56.064496 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a21c53a-9fb8-4f9f-9064-7ada9510ea20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba6d5b00cc14526819421a6a3b290fc, user_id=df925c3f452c4a7d99ba942e20607b20, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:56.067919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a21c53a-9fb8-4f9f-9064-7ada9510ea20 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:56.077247 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 944/1913] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:55 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:56.078383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1317d718-b04c-40ae-9ab0-110c191f3335 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:56.092401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 970/1914] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:55 2022] PUT /identity/v3/projects/5ba6d5b00cc14526819421a6a3b290fc/users/df925c3f452c4a7d99ba942e20607b20/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:56.092401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a21c53a-9fb8-4f9f-9064-7ada9510ea20 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:56.149636 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1317d718-b04c-40ae-9ab0-110c191f3335 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:56.157837 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:56.164441 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a21c53a-9fb8-4f9f-9064-7ada9510ea20 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:56.169639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e8b3c-cc05-408a-a104-d99229be1092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:56.170258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e8b3c-cc05-408a-a104-d99229be1092 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:56.170545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e8b3c-cc05-408a-a104-d99229be1092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:56.170822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e8b3c-cc05-408a-a104-d99229be1092 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:56.171489 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587e8b3c-cc05-408a-a104-d99229be1092 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:56.173456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587e8b3c-cc05-408a-a104-d99229be1092 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:56.195195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 971/1915] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:56 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:56.197189 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036f4c7-d62d-4913-9753-05f2911588af None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:56.197610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036f4c7-d62d-4913-9753-05f2911588af None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:56.198014 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036f4c7-d62d-4913-9753-05f2911588af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:56.221318 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3a633-35ed-4503-9722-af0e60bf7501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:56.221951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3a633-35ed-4503-9722-af0e60bf7501 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:56.224563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3a633-35ed-4503-9722-af0e60bf7501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:56.225269 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3a633-35ed-4503-9722-af0e60bf7501 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:56.227236 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf3a633-35ed-4503-9722-af0e60bf7501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:56.227864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1036f4c7-d62d-4913-9753-05f2911588af None None] MFA Rules not processed for user `df925c3f452c4a7d99ba942e20607b20`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:56.229132 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1036f4c7-d62d-4913-9753-05f2911588af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:56.229298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf3a633-35ed-4503-9722-af0e60bf7501 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:56.257395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 945/1916] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:56.258476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3a633-35ed-4503-9722-af0e60bf7501 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:56.281840 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3a633-35ed-4503-9722-af0e60bf7501 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:56.304090 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11f523-ff34-4feb-b0a6-5e254db94049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:56.305532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11f523-ff34-4feb-b0a6-5e254db94049 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:56.307382 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11f523-ff34-4feb-b0a6-5e254db94049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:56.307753 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11f523-ff34-4feb-b0a6-5e254db94049 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:56.311531 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11f523-ff34-4feb-b0a6-5e254db94049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:56.313575 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11f523-ff34-4feb-b0a6-5e254db94049 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:56.336513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 972/1917] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:56.338409 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1036f4c7-d62d-4913-9753-05f2911588af None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:56.346642 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 946/1918] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:56.350703 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11f523-ff34-4feb-b0a6-5e254db94049 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:56.354556 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1036f4c7-d62d-4913-9753-05f2911588af None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:56.362254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11f523-ff34-4feb-b0a6-5e254db94049 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:56.367075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03224011-30c5-4cb3-a70e-b68c66926752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:56.367893 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03224011-30c5-4cb3-a70e-b68c66926752 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:56.368073 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6cd19-a8b2-46c2-bd97-ab16eabd2a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:56.369222 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03224011-30c5-4cb3-a70e-b68c66926752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:56.369884 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03224011-30c5-4cb3-a70e-b68c66926752 admin admin] PATH_INFO: `/v3/projects/05076cfafedc42a792b60a3de8775728/users/186500f6f7bb4836a2c25192a5c4102f/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:56.370894 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6cd19-a8b2-46c2-bd97-ab16eabd2a26 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:56.371180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6cd19-a8b2-46c2-bd97-ab16eabd2a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:56.371885 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6cd19-a8b2-46c2-bd97-ab16eabd2a26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:56.373601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03224011-30c5-4cb3-a70e-b68c66926752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05076cfafedc42a792b60a3de8775728, user_id=186500f6f7bb4836a2c25192a5c4102f, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:56.374243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6cd19-a8b2-46c2-bd97-ab16eabd2a26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:56.376567 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6cd19-a8b2-46c2-bd97-ab16eabd2a26 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:56.377267 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03224011-30c5-4cb3-a70e-b68c66926752 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:56.396407 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 973/1919] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:56 2022] PUT /identity/v3/projects/05076cfafedc42a792b60a3de8775728/users/186500f6f7bb4836a2c25192a5c4102f/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:56.396407 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03224011-30c5-4cb3-a70e-b68c66926752 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:56.437219 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 947/1920] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:56 2022] POST /identity/v3/projects => generated 383 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:56.440204 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594cd5f-0aba-4118-ab77-6e56ed8c4f14 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:56.444773 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594cd5f-0aba-4118-ab77-6e56ed8c4f14 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:56.444999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594cd5f-0aba-4118-ab77-6e56ed8c4f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:56.509195 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-d594cd5f-0aba-4118-ab77-6e56ed8c4f14 None None] MFA Rules not processed for user `186500f6f7bb4836a2c25192a5c4102f`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:56.521800 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d594cd5f-0aba-4118-ab77-6e56ed8c4f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:56.539068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03224011-30c5-4cb3-a70e-b68c66926752 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:56.561603 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:56.617071 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 948/1921] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:56 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:56.618117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594cd5f-0aba-4118-ab77-6e56ed8c4f14 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:56.674807 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403c9d0-d528-45c7-a935-70b37150640f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:56.675879 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403c9d0-d528-45c7-a935-70b37150640f admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:56.676518 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403c9d0-d528-45c7-a935-70b37150640f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:56.676980 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403c9d0-d528-45c7-a935-70b37150640f admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:56.678158 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5403c9d0-d528-45c7-a935-70b37150640f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:56.680718 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5403c9d0-d528-45c7-a935-70b37150640f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:56.689705 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594cd5f-0aba-4118-ab77-6e56ed8c4f14 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:56.695357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1d843-7163-4e47-bc37-42f9ad53f840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:56.697253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1d843-7163-4e47-bc37-42f9ad53f840 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:56.697253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1d843-7163-4e47-bc37-42f9ad53f840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:56.697253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1d843-7163-4e47-bc37-42f9ad53f840 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:56.697523 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1d843-7163-4e47-bc37-42f9ad53f840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:56.699356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1d843-7163-4e47-bc37-42f9ad53f840 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:56.711495 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 974/1922] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:56.717934 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403c9d0-d528-45c7-a935-70b37150640f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:56.721063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5403c9d0-d528-45c7-a935-70b37150640f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:56.731739 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 949/1923] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:56 2022] POST /identity/v3/users => generated 298 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:56.732712 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1d843-7163-4e47-bc37-42f9ad53f840 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:56.741973 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:56.745782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1d843-7163-4e47-bc37-42f9ad53f840 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:56.751044 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7187d469-8286-48f0-bc30-03ceffc68e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:56.752071 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7187d469-8286-48f0-bc30-03ceffc68e2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:56.752382 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7187d469-8286-48f0-bc30-03ceffc68e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:56.752706 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7187d469-8286-48f0-bc30-03ceffc68e2d admin admin] PATH_INFO: `/v3/projects/185fdd7d6da54802ae25d4ff026c3ccd/users/bf2295631e384f73b7f5f266a2f2b65a/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:56.773458 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7187d469-8286-48f0-bc30-03ceffc68e2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=185fdd7d6da54802ae25d4ff026c3ccd, user_id=bf2295631e384f73b7f5f266a2f2b65a, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:56.776342 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7187d469-8286-48f0-bc30-03ceffc68e2d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:56.797188 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 950/1924] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:56 2022] PUT /identity/v3/projects/185fdd7d6da54802ae25d4ff026c3ccd/users/bf2295631e384f73b7f5f266a2f2b65a/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:56.798076 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7187d469-8286-48f0-bc30-03ceffc68e2d admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:56.873544 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403c9d0-d528-45c7-a935-70b37150640f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:56.913828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7187d469-8286-48f0-bc30-03ceffc68e2d admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:56.921025 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d4790-4a43-414e-a4c8-c320d058472b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:56.922479 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d4790-4a43-414e-a4c8-c320d058472b admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:56.922772 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d4790-4a43-414e-a4c8-c320d058472b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:56.923124 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d4790-4a43-414e-a4c8-c320d058472b admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:56.924054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3d4790-4a43-414e-a4c8-c320d058472b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:56.927436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3d4790-4a43-414e-a4c8-c320d058472b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:56.944114 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 951/1925] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:56.946217 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f031f-e955-4084-b6a5-fce5698ea161 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:56.946517 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f031f-e955-4084-b6a5-fce5698ea161 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:56.946877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f031f-e955-4084-b6a5-fce5698ea161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:56.964903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08995b14-647d-4a18-9174-dac52a26886c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:56.967973 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08995b14-647d-4a18-9174-dac52a26886c service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:56.968721 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08995b14-647d-4a18-9174-dac52a26886c service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:56.969647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08995b14-647d-4a18-9174-dac52a26886c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:56.994152 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-700f031f-e955-4084-b6a5-fce5698ea161 None None] MFA Rules not processed for user `bf2295631e384f73b7f5f266a2f2b65a`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:56.995927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-700f031f-e955-4084-b6a5-fce5698ea161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:57.034410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:57.145453 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08995b14-647d-4a18-9174-dac52a26886c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:57.146910 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08995b14-647d-4a18-9174-dac52a26886c service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:57.155925 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 952/1926] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:57.156654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700f031f-e955-4084-b6a5-fce5698ea161 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:57.171558 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 975/1927] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:56 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:57.173907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08995b14-647d-4a18-9174-dac52a26886c service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:57.178382 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700f031f-e955-4084-b6a5-fce5698ea161 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:57.192117 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08995b14-647d-4a18-9174-dac52a26886c service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:57.194137 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa48c1-a108-44d5-b097-4100fce71841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:57.194901 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa48c1-a108-44d5-b097-4100fce71841 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:57.195256 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa48c1-a108-44d5-b097-4100fce71841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:57.195776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa48c1-a108-44d5-b097-4100fce71841 admin admin] PATH_INFO: `/v3/projects/703700e39c414c5b87be07aa8d07bf80/users/0efcd370bf9b4a51b746b95a88a2b0ac/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:57.209797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965253d-e44e-48b9-a132-3ff6b8e844ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:57.210529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965253d-e44e-48b9-a132-3ff6b8e844ac admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:57.210880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965253d-e44e-48b9-a132-3ff6b8e844ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:57.211264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965253d-e44e-48b9-a132-3ff6b8e844ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:57.211961 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965253d-e44e-48b9-a132-3ff6b8e844ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:57.215978 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965253d-e44e-48b9-a132-3ff6b8e844ac admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:57.229742 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa48c1-a108-44d5-b097-4100fce71841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=703700e39c414c5b87be07aa8d07bf80, user_id=0efcd370bf9b4a51b746b95a88a2b0ac, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:57.240619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa48c1-a108-44d5-b097-4100fce71841 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:57.280433 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 953/1928] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:57 2022] PUT /identity/v3/projects/703700e39c414c5b87be07aa8d07bf80/users/0efcd370bf9b4a51b746b95a88a2b0ac/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:57.280601 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa48c1-a108-44d5-b097-4100fce71841 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:57.289013 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 976/1929] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:30:57 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:57.289747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965253d-e44e-48b9-a132-3ff6b8e844ac admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:57.390610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:57.436594 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa48c1-a108-44d5-b097-4100fce71841 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:57.438818 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965253d-e44e-48b9-a132-3ff6b8e844ac admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:57.454355 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad3e53-06cc-4d06-91aa-fff5fcae0d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:57.455373 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad3e53-06cc-4d06-91aa-fff5fcae0d99 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:57.455809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad3e53-06cc-4d06-91aa-fff5fcae0d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:57.456187 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad3e53-06cc-4d06-91aa-fff5fcae0d99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:57.458231 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fad3e53-06cc-4d06-91aa-fff5fcae0d99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:57.462547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fad3e53-06cc-4d06-91aa-fff5fcae0d99 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:57.495307 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 977/1930] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:57.496694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad3e53-06cc-4d06-91aa-fff5fcae0d99 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:57.510427 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad3e53-06cc-4d06-91aa-fff5fcae0d99 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:57.515689 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b7767-7afd-4f23-b707-6443635e195f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:57.516741 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b7767-7afd-4f23-b707-6443635e195f admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:57.517986 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b7767-7afd-4f23-b707-6443635e195f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:57.518423 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b7767-7afd-4f23-b707-6443635e195f admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:57.520352 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b7767-7afd-4f23-b707-6443635e195f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:57.523507 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b7767-7afd-4f23-b707-6443635e195f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:57.569422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a131619-f4c6-4ceb-ade4-c03ebde58757 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:57.571098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a131619-f4c6-4ceb-ade4-c03ebde58757 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:57.571210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 978/1931] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:30:57 2022] POST /identity/v3/users => generated 293 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:30:57.571610 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a131619-f4c6-4ceb-ade4-c03ebde58757 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:57.572059 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a131619-f4c6-4ceb-ade4-c03ebde58757 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:57.577269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b7767-7afd-4f23-b707-6443635e195f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:57.578209 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0a131619-f4c6-4ceb-ade4-c03ebde58757 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:57.595412 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b7767-7afd-4f23-b707-6443635e195f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:57.603252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9f076-4371-4d0e-88e7-669204414d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:57.604086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9f076-4371-4d0e-88e7-669204414d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:57.604530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9f076-4371-4d0e-88e7-669204414d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:57.605213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9f076-4371-4d0e-88e7-669204414d16 admin admin] PATH_INFO: `/v3/projects/703700e39c414c5b87be07aa8d07bf80/users/0efcd370bf9b4a51b746b95a88a2b0ac/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:57.607906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9f076-4371-4d0e-88e7-669204414d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=703700e39c414c5b87be07aa8d07bf80, user_id=0efcd370bf9b4a51b746b95a88a2b0ac, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:57.611217 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9f076-4371-4d0e-88e7-669204414d16 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:57.627742 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 979/1932] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:57 2022] PUT /identity/v3/projects/703700e39c414c5b87be07aa8d07bf80/users/0efcd370bf9b4a51b746b95a88a2b0ac/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:57.640619 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9f076-4371-4d0e-88e7-669204414d16 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:57.719835 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a131619-f4c6-4ceb-ade4-c03ebde58757 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:57.721961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a131619-f4c6-4ceb-ade4-c03ebde58757 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:57.735937 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 954/1933] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:57 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:57.737328 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa891070-64fc-4068-ba1c-b376517c0177 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:57.737619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa891070-64fc-4068-ba1c-b376517c0177 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:57.738721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa891070-64fc-4068-ba1c-b376517c0177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:57.748980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:57.758304 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9f076-4371-4d0e-88e7-669204414d16 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:57.763952 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac06d39-2337-425e-9b71-d529af1cfb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:57.764647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac06d39-2337-425e-9b71-d529af1cfb42 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:57.765348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac06d39-2337-425e-9b71-d529af1cfb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:57.765656 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac06d39-2337-425e-9b71-d529af1cfb42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:57.766386 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac06d39-2337-425e-9b71-d529af1cfb42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:57.769636 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac06d39-2337-425e-9b71-d529af1cfb42 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:57.779879 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-fa891070-64fc-4068-ba1c-b376517c0177 None None] MFA Rules not processed for user `0efcd370bf9b4a51b746b95a88a2b0ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:57.783904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 980/1934] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:30:57.784037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fa891070-64fc-4068-ba1c-b376517c0177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:57.789004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac06d39-2337-425e-9b71-d529af1cfb42 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:57.806003 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac06d39-2337-425e-9b71-d529af1cfb42 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:57.812015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ca0df-24f5-47fa-85d5-3a0a80e88338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:57.813000 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ca0df-24f5-47fa-85d5-3a0a80e88338 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:57.813415 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ca0df-24f5-47fa-85d5-3a0a80e88338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:57.813818 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ca0df-24f5-47fa-85d5-3a0a80e88338 admin admin] PATH_INFO: `/v3/projects/07cf2f882e0842bd9e2b0c7251662209/users/36b92f88a032415aa32271f60f2537b8/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:57.840194 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1ca0df-24f5-47fa-85d5-3a0a80e88338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07cf2f882e0842bd9e2b0c7251662209, user_id=36b92f88a032415aa32271f60f2537b8, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:57.844957 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1ca0df-24f5-47fa-85d5-3a0a80e88338 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:57.868914 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 981/1935] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:57 2022] PUT /identity/v3/projects/07cf2f882e0842bd9e2b0c7251662209/users/36b92f88a032415aa32271f60f2537b8/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:57.873169 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ca0df-24f5-47fa-85d5-3a0a80e88338 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:57.886479 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 955/1936] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:57.900394 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa891070-64fc-4068-ba1c-b376517c0177 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:57.902214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fa891070-64fc-4068-ba1c-b376517c0177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:58.020587 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ca0df-24f5-47fa-85d5-3a0a80e88338 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:58.024435 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa891070-64fc-4068-ba1c-b376517c0177 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:58.027703 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96ae16-904f-4663-85f8-241bd1221c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:58.028614 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96ae16-904f-4663-85f8-241bd1221c3c admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:58.030099 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96ae16-904f-4663-85f8-241bd1221c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:58.030528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96ae16-904f-4663-85f8-241bd1221c3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:58.031816 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96ae16-904f-4663-85f8-241bd1221c3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:58.035739 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96ae16-904f-4663-85f8-241bd1221c3c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:58.067596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 982/1937] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 12:30:58.072769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96ae16-904f-4663-85f8-241bd1221c3c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:58.090557 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96ae16-904f-4663-85f8-241bd1221c3c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:58.110950 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d487e54-df0e-4a2a-9c2e-8104e48e19b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:58.111641 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d487e54-df0e-4a2a-9c2e-8104e48e19b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:58.111922 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d487e54-df0e-4a2a-9c2e-8104e48e19b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:58.112197 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d487e54-df0e-4a2a-9c2e-8104e48e19b2 admin admin] PATH_INFO: `/v3/projects/07cf2f882e0842bd9e2b0c7251662209/users/36b92f88a032415aa32271f60f2537b8/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:58.114275 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d487e54-df0e-4a2a-9c2e-8104e48e19b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07cf2f882e0842bd9e2b0c7251662209, user_id=36b92f88a032415aa32271f60f2537b8, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:58.117222 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d487e54-df0e-4a2a-9c2e-8104e48e19b2 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:58.133817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511a6e9-a80a-4acc-9ff6-e4b2ee311a97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:58.138674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511a6e9-a80a-4acc-9ff6-e4b2ee311a97 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:58.138674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511a6e9-a80a-4acc-9ff6-e4b2ee311a97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:58.140696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511a6e9-a80a-4acc-9ff6-e4b2ee311a97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:58.164721 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 983/1938] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:30:58 2022] PUT /identity/v3/projects/07cf2f882e0842bd9e2b0c7251662209/users/36b92f88a032415aa32271f60f2537b8/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:58.171062 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14bf2c-9227-400b-ab70-ccaab11ceb3b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:58.171542 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14bf2c-9227-400b-ab70-ccaab11ceb3b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:58.172751 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14bf2c-9227-400b-ab70-ccaab11ceb3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:58.204362 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:58.218999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3a14bf2c-9227-400b-ab70-ccaab11ceb3b None None] MFA Rules not processed for user `36b92f88a032415aa32271f60f2537b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:58.220542 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3a14bf2c-9227-400b-ab70-ccaab11ceb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:58.257573 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:58.260236 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1511a6e9-a80a-4acc-9ff6-e4b2ee311a97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:58.262262 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1511a6e9-a80a-4acc-9ff6-e4b2ee311a97 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:58.275025 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 956/1939] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:57 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:30:58.305739 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 984/1940] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:58.318268 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511a6e9-a80a-4acc-9ff6-e4b2ee311a97 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:58.327941 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511a6e9-a80a-4acc-9ff6-e4b2ee311a97 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:58.383281 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14bf2c-9227-400b-ab70-ccaab11ceb3b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:58.391011 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99af9d-742b-40b4-8bce-45a85b512313 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:58.391632 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99af9d-742b-40b4-8bce-45a85b512313 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:58.391909 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99af9d-742b-40b4-8bce-45a85b512313 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:58.392179 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99af9d-742b-40b4-8bce-45a85b512313 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:58.402299 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99af9d-742b-40b4-8bce-45a85b512313 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:58.405076 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99af9d-742b-40b4-8bce-45a85b512313 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:58.417001 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 957/1941] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:58.465205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14bf2c-9227-400b-ab70-ccaab11ceb3b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:58.469710 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484b20b-a278-4488-a6e5-4530387e3095 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:58.470737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484b20b-a278-4488-a6e5-4530387e3095 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:58.470737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484b20b-a278-4488-a6e5-4530387e3095 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:58.471014 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484b20b-a278-4488-a6e5-4530387e3095 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:58.570494 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484b20b-a278-4488-a6e5-4530387e3095 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:58.572201 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484b20b-a278-4488-a6e5-4530387e3095 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:58.585021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 985/1942] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:58.663775 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99af9d-742b-40b4-8bce-45a85b512313 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:58.665276 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-aa99af9d-742b-40b4-8bce-45a85b512313 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:58.782133 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99af9d-742b-40b4-8bce-45a85b512313 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:58.790525 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d741ac-1e80-4def-acec-f9be3ede173a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:58.792047 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d741ac-1e80-4def-acec-f9be3ede173a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:58.792650 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d741ac-1e80-4def-acec-f9be3ede173a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:58.793028 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d741ac-1e80-4def-acec-f9be3ede173a admin admin] PATH_INFO: `/v3/users/2fcacf154a864a2e921864e23dcebfd8` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:58.795236 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d741ac-1e80-4def-acec-f9be3ede173a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fcacf154a864a2e921864e23dcebfd8)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:58.805238 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d741ac-1e80-4def-acec-f9be3ede173a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:58.909086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e4d741ac-1e80-4def-acec-f9be3ede173a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fcacf154a864a2e921864e23dcebfd8'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:58.919263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e4d741ac-1e80-4def-acec-f9be3ede173a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fcacf154a864a2e921864e23dcebfd8'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:58.944706 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e4d741ac-1e80-4def-acec-f9be3ede173a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fcacf154a864a2e921864e23dcebfd8'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:58.947633 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e4d741ac-1e80-4def-acec-f9be3ede173a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fcacf154a864a2e921864e23dcebfd8'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:58.948765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e4d741ac-1e80-4def-acec-f9be3ede173a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fcacf154a864a2e921864e23dcebfd8'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:58.970557 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 958/1943] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:30:58 2022] DELETE /identity/v3/users/2fcacf154a864a2e921864e23dcebfd8 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:58.975209 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04991cb0-56d7-47e5-ad93-4880954ff472 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:58.975498 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04991cb0-56d7-47e5-ad93-4880954ff472 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:58.975903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04991cb0-56d7-47e5-ad93-4880954ff472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:59.007690 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-04991cb0-56d7-47e5-ad93-4880954ff472 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:59.012143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-04991cb0-56d7-47e5-ad93-4880954ff472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:59.160727 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:59.192419 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d741ac-1e80-4def-acec-f9be3ede173a admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:59.193955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e4d741ac-1e80-4def-acec-f9be3ede173a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:59.240679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 986/1944] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:59.246954 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:30:59.260156 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04991cb0-56d7-47e5-ad93-4880954ff472 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:59.262028 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-04991cb0-56d7-47e5-ad93-4880954ff472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:59.325919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d741ac-1e80-4def-acec-f9be3ede173a admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:59.331147 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb3e2f-2ecf-47fd-90a4-cd2589cd278e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:59.332376 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb3e2f-2ecf-47fd-90a4-cd2589cd278e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:59.332663 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb3e2f-2ecf-47fd-90a4-cd2589cd278e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:59.332937 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb3e2f-2ecf-47fd-90a4-cd2589cd278e admin admin] PATH_INFO: `/v3/users/bf2295631e384f73b7f5f266a2f2b65a` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:59.334852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb3e2f-2ecf-47fd-90a4-cd2589cd278e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf2295631e384f73b7f5f266a2f2b65a)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:59.338362 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb3e2f-2ecf-47fd-90a4-cd2589cd278e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:59.390739 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04991cb0-56d7-47e5-ad93-4880954ff472 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:59.400938 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b1dcd-f54e-4c46-991f-de46425bf8d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:59.401601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b1dcd-f54e-4c46-991f-de46425bf8d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:59.401884 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b1dcd-f54e-4c46-991f-de46425bf8d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:59.402159 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b1dcd-f54e-4c46-991f-de46425bf8d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:59.423719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b1dcd-f54e-4c46-991f-de46425bf8d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:59.425378 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b1dcd-f54e-4c46-991f-de46425bf8d1 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:59.429045 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e5eb3e2f-2ecf-47fd-90a4-cd2589cd278e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf2295631e384f73b7f5f266a2f2b65a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:59.438480 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e5eb3e2f-2ecf-47fd-90a4-cd2589cd278e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf2295631e384f73b7f5f266a2f2b65a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:59.449079 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 987/1945] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:30:59.454802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e5eb3e2f-2ecf-47fd-90a4-cd2589cd278e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf2295631e384f73b7f5f266a2f2b65a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:59.455560 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e5eb3e2f-2ecf-47fd-90a4-cd2589cd278e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf2295631e384f73b7f5f266a2f2b65a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:59.456801 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e5eb3e2f-2ecf-47fd-90a4-cd2589cd278e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf2295631e384f73b7f5f266a2f2b65a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:30:59.485207 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 959/1946] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:30:59 2022] DELETE /identity/v3/users/bf2295631e384f73b7f5f266a2f2b65a => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:30:59.490922 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0946b-9b45-45a7-a589-b2bf9ad8208a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:59.491487 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0946b-9b45-45a7-a589-b2bf9ad8208a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:59.491487 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0946b-9b45-45a7-a589-b2bf9ad8208a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:59.517874 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-30f0946b-9b45-45a7-a589-b2bf9ad8208a None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:30:59.519098 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-30f0946b-9b45-45a7-a589-b2bf9ad8208a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:59.608538 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 988/1947] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:30:59.619249 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb3e2f-2ecf-47fd-90a4-cd2589cd278e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:59.620757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e5eb3e2f-2ecf-47fd-90a4-cd2589cd278e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:59.689348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb3e2f-2ecf-47fd-90a4-cd2589cd278e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:59.693960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe363f00-c7f5-4e87-8832-cdb43731e999 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:59.694657 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe363f00-c7f5-4e87-8832-cdb43731e999 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:59.696404 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe363f00-c7f5-4e87-8832-cdb43731e999 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:59.696689 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe363f00-c7f5-4e87-8832-cdb43731e999 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:59.709572 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe363f00-c7f5-4e87-8832-cdb43731e999 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:59.711452 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe363f00-c7f5-4e87-8832-cdb43731e999 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:30:59.723361 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 960/1948] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:30:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:30:59.900904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0946b-9b45-45a7-a589-b2bf9ad8208a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:30:59.903345 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-30f0946b-9b45-45a7-a589-b2bf9ad8208a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:30:59.922332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0946b-9b45-45a7-a589-b2bf9ad8208a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:30:59.932195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0026c95-e571-45e3-8790-3b06474571f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:30:59.932937 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0026c95-e571-45e3-8790-3b06474571f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:30:59.933219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0026c95-e571-45e3-8790-3b06474571f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:30:59.933523 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0026c95-e571-45e3-8790-3b06474571f1 admin admin] PATH_INFO: `/v3/projects/94a888bc2de340b08d6ffd860e11b215` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:30:59.935172 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0026c95-e571-45e3-8790-3b06474571f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94a888bc2de340b08d6ffd860e11b215)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:30:59.937436 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0026c95-e571-45e3-8790-3b06474571f1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:00.046854 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e0026c95-e571-45e3-8790-3b06474571f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94a888bc2de340b08d6ffd860e11b215', 'request_id': 'req-e0026c95-e571-45e3-8790-3b06474571f1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:00.055328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e0026c95-e571-45e3-8790-3b06474571f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94a888bc2de340b08d6ffd860e11b215', 'request_id': 'req-e0026c95-e571-45e3-8790-3b06474571f1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:00.067346 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e0026c95-e571-45e3-8790-3b06474571f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94a888bc2de340b08d6ffd860e11b215', 'request_id': 'req-e0026c95-e571-45e3-8790-3b06474571f1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:00.087174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e0026c95-e571-45e3-8790-3b06474571f1 admin admin] The token cache is being invalidate because project 94a888bc2de340b08d6ffd860e11b215 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:31:00.087774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e0026c95-e571-45e3-8790-3b06474571f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:00.089729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e0026c95-e571-45e3-8790-3b06474571f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:00.091897 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 989/1949] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:30:59 2022] DELETE /identity/v3/projects/94a888bc2de340b08d6ffd860e11b215 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:00.095861 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe363f00-c7f5-4e87-8832-cdb43731e999 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:00.110875 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe363f00-c7f5-4e87-8832-cdb43731e999 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:00.168031 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:00.200067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4659c-8b4d-4d2c-b3be-120687001570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:00.201482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4659c-8b4d-4d2c-b3be-120687001570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:00.202044 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4659c-8b4d-4d2c-b3be-120687001570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:00.202865 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4659c-8b4d-4d2c-b3be-120687001570 admin admin] PATH_INFO: `/v3/users/96014290ba4049ad85def453315128df` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:00.205387 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4659c-8b4d-4d2c-b3be-120687001570 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96014290ba4049ad85def453315128df)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:00.209517 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4659c-8b4d-4d2c-b3be-120687001570 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:00.318988 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4db4659c-8b4d-4d2c-b3be-120687001570 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96014290ba4049ad85def453315128df'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:00.332558 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0026c95-e571-45e3-8790-3b06474571f1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:00.334410 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e0026c95-e571-45e3-8790-3b06474571f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:00.346126 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4db4659c-8b4d-4d2c-b3be-120687001570 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96014290ba4049ad85def453315128df'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:00.368893 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0026c95-e571-45e3-8790-3b06474571f1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:00.375766 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331abed5-7cc3-47e9-9529-edebb44bbe47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:00.376913 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331abed5-7cc3-47e9-9529-edebb44bbe47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:00.377442 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331abed5-7cc3-47e9-9529-edebb44bbe47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:00.377851 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331abed5-7cc3-47e9-9529-edebb44bbe47 admin admin] PATH_INFO: `/v3/projects/185fdd7d6da54802ae25d4ff026c3ccd` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:00.380396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331abed5-7cc3-47e9-9529-edebb44bbe47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=185fdd7d6da54802ae25d4ff026c3ccd)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:00.382624 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331abed5-7cc3-47e9-9529-edebb44bbe47 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:00.383951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4db4659c-8b4d-4d2c-b3be-120687001570 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96014290ba4049ad85def453315128df'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:00.385127 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4db4659c-8b4d-4d2c-b3be-120687001570 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96014290ba4049ad85def453315128df'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:00.385808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4db4659c-8b4d-4d2c-b3be-120687001570 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96014290ba4049ad85def453315128df'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:00.408140 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 961/1950] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:31:00 2022] DELETE /identity/v3/users/96014290ba4049ad85def453315128df => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:00.420582 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:00.500085 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-331abed5-7cc3-47e9-9529-edebb44bbe47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '185fdd7d6da54802ae25d4ff026c3ccd', 'request_id': 'req-331abed5-7cc3-47e9-9529-edebb44bbe47'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:00.512461 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-331abed5-7cc3-47e9-9529-edebb44bbe47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '185fdd7d6da54802ae25d4ff026c3ccd', 'request_id': 'req-331abed5-7cc3-47e9-9529-edebb44bbe47'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:00.529539 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-331abed5-7cc3-47e9-9529-edebb44bbe47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '185fdd7d6da54802ae25d4ff026c3ccd', 'request_id': 'req-331abed5-7cc3-47e9-9529-edebb44bbe47'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:00.554460 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-331abed5-7cc3-47e9-9529-edebb44bbe47 admin admin] The token cache is being invalidate because project 185fdd7d6da54802ae25d4ff026c3ccd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:31:00.555210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-331abed5-7cc3-47e9-9529-edebb44bbe47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:00.556155 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-331abed5-7cc3-47e9-9529-edebb44bbe47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:00.558203 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 990/1951] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:31:00 2022] DELETE /identity/v3/projects/185fdd7d6da54802ae25d4ff026c3ccd => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:31:00.562618 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4659c-8b4d-4d2c-b3be-120687001570 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:00.564586 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4db4659c-8b4d-4d2c-b3be-120687001570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:00.668402 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4659c-8b4d-4d2c-b3be-120687001570 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:00.758153 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40404f-559f-4102-b99c-29c487fb0e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:00.758858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40404f-559f-4102-b99c-29c487fb0e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:00.759521 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40404f-559f-4102-b99c-29c487fb0e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:00.759521 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40404f-559f-4102-b99c-29c487fb0e4b admin admin] PATH_INFO: `/v3/users/36b92f88a032415aa32271f60f2537b8` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:00.761013 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d40404f-559f-4102-b99c-29c487fb0e4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=36b92f88a032415aa32271f60f2537b8)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:00.763302 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d40404f-559f-4102-b99c-29c487fb0e4b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:00.773037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:00.830116 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8d40404f-559f-4102-b99c-29c487fb0e4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36b92f88a032415aa32271f60f2537b8'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:00.838382 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8d40404f-559f-4102-b99c-29c487fb0e4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36b92f88a032415aa32271f60f2537b8'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:00.860825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8d40404f-559f-4102-b99c-29c487fb0e4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36b92f88a032415aa32271f60f2537b8'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:00.861721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8d40404f-559f-4102-b99c-29c487fb0e4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36b92f88a032415aa32271f60f2537b8'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:00.865224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8d40404f-559f-4102-b99c-29c487fb0e4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36b92f88a032415aa32271f60f2537b8'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:00.896666 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 962/1952] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:31:00 2022] DELETE /identity/v3/users/36b92f88a032415aa32271f60f2537b8 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:01.321142 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331abed5-7cc3-47e9-9529-edebb44bbe47 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:01.322695 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-331abed5-7cc3-47e9-9529-edebb44bbe47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:01.377277 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331abed5-7cc3-47e9-9529-edebb44bbe47 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:01.584468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c6746-c276-4634-8621-c579abf4f5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:01.588132 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c6746-c276-4634-8621-c579abf4f5d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:01.591719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c6746-c276-4634-8621-c579abf4f5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:01.592713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c6746-c276-4634-8621-c579abf4f5d4 admin admin] PATH_INFO: `/v3/projects/4c250878bd204cb49f4458ace310ae9f` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:01.603671 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c6746-c276-4634-8621-c579abf4f5d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c250878bd204cb49f4458ace310ae9f)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:01.604199 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40404f-559f-4102-b99c-29c487fb0e4b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:01.605851 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8d40404f-559f-4102-b99c-29c487fb0e4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:01.606585 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c6746-c276-4634-8621-c579abf4f5d4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:01.625931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40404f-559f-4102-b99c-29c487fb0e4b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:01.632043 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791915d-1b72-4b75-9e08-64c1f5b91807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:01.633583 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791915d-1b72-4b75-9e08-64c1f5b91807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:01.634211 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791915d-1b72-4b75-9e08-64c1f5b91807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:01.634652 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791915d-1b72-4b75-9e08-64c1f5b91807 admin admin] PATH_INFO: `/v3/users/df925c3f452c4a7d99ba942e20607b20` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:01.636136 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791915d-1b72-4b75-9e08-64c1f5b91807 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df925c3f452c4a7d99ba942e20607b20)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:01.638331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791915d-1b72-4b75-9e08-64c1f5b91807 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:01.685117 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4c3c6746-c276-4634-8621-c579abf4f5d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c250878bd204cb49f4458ace310ae9f', 'request_id': 'req-4c3c6746-c276-4634-8621-c579abf4f5d4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:01.699689 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4c3c6746-c276-4634-8621-c579abf4f5d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c250878bd204cb49f4458ace310ae9f', 'request_id': 'req-4c3c6746-c276-4634-8621-c579abf4f5d4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:01.721445 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f791915d-1b72-4b75-9e08-64c1f5b91807 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df925c3f452c4a7d99ba942e20607b20'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:01.726039 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4c3c6746-c276-4634-8621-c579abf4f5d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c250878bd204cb49f4458ace310ae9f', 'request_id': 'req-4c3c6746-c276-4634-8621-c579abf4f5d4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:01.732577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f791915d-1b72-4b75-9e08-64c1f5b91807 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df925c3f452c4a7d99ba942e20607b20'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:01.750144 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f791915d-1b72-4b75-9e08-64c1f5b91807 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df925c3f452c4a7d99ba942e20607b20'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:01.750710 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4c3c6746-c276-4634-8621-c579abf4f5d4 admin admin] The token cache is being invalidate because project 4c250878bd204cb49f4458ace310ae9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:31:01.751005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f791915d-1b72-4b75-9e08-64c1f5b91807 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df925c3f452c4a7d99ba942e20607b20'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:01.751302 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4c3c6746-c276-4634-8621-c579abf4f5d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:01.751976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4c3c6746-c276-4634-8621-c579abf4f5d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:01.752557 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f791915d-1b72-4b75-9e08-64c1f5b91807 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df925c3f452c4a7d99ba942e20607b20'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:01.756717 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 991/1953] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:31:01 2022] DELETE /identity/v3/projects/4c250878bd204cb49f4458ace310ae9f => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:01.758356 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c6746-c276-4634-8621-c579abf4f5d4 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:01.760179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4c3c6746-c276-4634-8621-c579abf4f5d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:01.778090 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 963/1954] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:31:01 2022] DELETE /identity/v3/users/df925c3f452c4a7d99ba942e20607b20 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:01.780149 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791915d-1b72-4b75-9e08-64c1f5b91807 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:01.781904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f791915d-1b72-4b75-9e08-64c1f5b91807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:01.798907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:01.838730 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c6746-c276-4634-8621-c579abf4f5d4 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:01.856377 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:01.861273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791915d-1b72-4b75-9e08-64c1f5b91807 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:01.869139 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887af508-0008-4ce1-bb40-319bdc471fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:01.869821 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887af508-0008-4ce1-bb40-319bdc471fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:01.870406 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887af508-0008-4ce1-bb40-319bdc471fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:01.871103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887af508-0008-4ce1-bb40-319bdc471fde admin admin] PATH_INFO: `/v3/users/11a94f917f3445b4a1b19384b2fb1e5b` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:01.873383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887af508-0008-4ce1-bb40-319bdc471fde admin admin] RBAC: Authorizing `identity:delete_user(user_id=11a94f917f3445b4a1b19384b2fb1e5b)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:01.877890 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887af508-0008-4ce1-bb40-319bdc471fde admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:01.902830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66946464-fd70-40e7-b84d-ff66011b5a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:01.903481 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66946464-fd70-40e7-b84d-ff66011b5a40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:01.903759 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66946464-fd70-40e7-b84d-ff66011b5a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:01.904031 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66946464-fd70-40e7-b84d-ff66011b5a40 admin admin] PATH_INFO: `/v3/projects/07cf2f882e0842bd9e2b0c7251662209` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:01.905243 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66946464-fd70-40e7-b84d-ff66011b5a40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07cf2f882e0842bd9e2b0c7251662209)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:01.907550 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66946464-fd70-40e7-b84d-ff66011b5a40 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:01.932202 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-887af508-0008-4ce1-bb40-319bdc471fde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11a94f917f3445b4a1b19384b2fb1e5b'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:01.939318 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-887af508-0008-4ce1-bb40-319bdc471fde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11a94f917f3445b4a1b19384b2fb1e5b'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:01.952068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-887af508-0008-4ce1-bb40-319bdc471fde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11a94f917f3445b4a1b19384b2fb1e5b'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:01.952784 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-887af508-0008-4ce1-bb40-319bdc471fde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11a94f917f3445b4a1b19384b2fb1e5b'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:01.953392 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-887af508-0008-4ce1-bb40-319bdc471fde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11a94f917f3445b4a1b19384b2fb1e5b'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:01.955949 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-66946464-fd70-40e7-b84d-ff66011b5a40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07cf2f882e0842bd9e2b0c7251662209', 'request_id': 'req-66946464-fd70-40e7-b84d-ff66011b5a40'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:01.963234 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-66946464-fd70-40e7-b84d-ff66011b5a40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07cf2f882e0842bd9e2b0c7251662209', 'request_id': 'req-66946464-fd70-40e7-b84d-ff66011b5a40'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:01.967280 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 964/1955] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:31:01 2022] DELETE /identity/v3/users/11a94f917f3445b4a1b19384b2fb1e5b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:01.968627 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93a434-dfc1-4c83-a1fa-5ad8c8431dbf None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:01.968947 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93a434-dfc1-4c83-a1fa-5ad8c8431dbf None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:01.969196 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93a434-dfc1-4c83-a1fa-5ad8c8431dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:01.971409 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-66946464-fd70-40e7-b84d-ff66011b5a40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07cf2f882e0842bd9e2b0c7251662209', 'request_id': 'req-66946464-fd70-40e7-b84d-ff66011b5a40'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:01.983404 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-66946464-fd70-40e7-b84d-ff66011b5a40 admin admin] The token cache is being invalidate because project 07cf2f882e0842bd9e2b0c7251662209 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:31:01.983678 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-66946464-fd70-40e7-b84d-ff66011b5a40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:01.984372 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-66946464-fd70-40e7-b84d-ff66011b5a40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:01.986138 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 992/1956] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:31:01 2022] DELETE /identity/v3/projects/07cf2f882e0842bd9e2b0c7251662209 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:01.986856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66946464-fd70-40e7-b84d-ff66011b5a40 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:01.996150 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-5a93a434-dfc1-4c83-a1fa-5ad8c8431dbf None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:01.998226 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5a93a434-dfc1-4c83-a1fa-5ad8c8431dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:02.002841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-66946464-fd70-40e7-b84d-ff66011b5a40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:02.111962 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 965/1957] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:02.119538 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4de38e-fceb-4af6-84b0-b820a3898e63 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:02.120038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4de38e-fceb-4af6-84b0-b820a3898e63 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:02.120503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4de38e-fceb-4af6-84b0-b820a3898e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:02.124897 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66946464-fd70-40e7-b84d-ff66011b5a40 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:02.133236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f9f5c-cb44-46c2-b1e4-f83f54746650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:02.133998 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f9f5c-cb44-46c2-b1e4-f83f54746650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:02.134361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f9f5c-cb44-46c2-b1e4-f83f54746650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:02.134955 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f9f5c-cb44-46c2-b1e4-f83f54746650 admin admin] PATH_INFO: `/v3/users/eac21bfdfeb942dab5a049eb0f74a146` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:02.136438 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f9f5c-cb44-46c2-b1e4-f83f54746650 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eac21bfdfeb942dab5a049eb0f74a146)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:02.138747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f9f5c-cb44-46c2-b1e4-f83f54746650 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:02.147313 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:02.150508 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-6b4de38e-fceb-4af6-84b0-b820a3898e63 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:02.152647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6b4de38e-fceb-4af6-84b0-b820a3898e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:02.164342 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 966/1958] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:02.172055 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4de38e-fceb-4af6-84b0-b820a3898e63 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:02.174537 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6b4de38e-fceb-4af6-84b0-b820a3898e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:02.223618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-df8f9f5c-cb44-46c2-b1e4-f83f54746650 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eac21bfdfeb942dab5a049eb0f74a146'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:02.240031 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-df8f9f5c-cb44-46c2-b1e4-f83f54746650 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eac21bfdfeb942dab5a049eb0f74a146'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:02.241018 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-df8f9f5c-cb44-46c2-b1e4-f83f54746650 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eac21bfdfeb942dab5a049eb0f74a146'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:02.241638 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-df8f9f5c-cb44-46c2-b1e4-f83f54746650 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eac21bfdfeb942dab5a049eb0f74a146'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:02.248429 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4de38e-fceb-4af6-84b0-b820a3898e63 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:02.251154 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-df8f9f5c-cb44-46c2-b1e4-f83f54746650 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eac21bfdfeb942dab5a049eb0f74a146'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:02.252646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f23fd5-c583-466e-bd75-2d87690a9221 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:02.253865 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f23fd5-c583-466e-bd75-2d87690a9221 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:02.254846 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f23fd5-c583-466e-bd75-2d87690a9221 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:02.255116 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f23fd5-c583-466e-bd75-2d87690a9221 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:02.257603 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b0f23fd5-c583-466e-bd75-2d87690a9221 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:02.268544 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 993/1959] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:31:01 2022] DELETE /identity/v3/users/eac21bfdfeb942dab5a049eb0f74a146 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:02.268544 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2106b8-ca5a-4ffa-8f90-d2cea1ea792f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:02.268544 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2106b8-ca5a-4ffa-8f90-d2cea1ea792f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:02.268544 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2106b8-ca5a-4ffa-8f90-d2cea1ea792f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:02.271803 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f23fd5-c583-466e-bd75-2d87690a9221 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:02.273236 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f23fd5-c583-466e-bd75-2d87690a9221 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:02.285304 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 967/1960] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:31:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:31:02.290582 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6a2106b8-ca5a-4ffa-8f90-d2cea1ea792f None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:02.290760 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3f599-3375-46d6-a57d-7a92116ca8a9 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:02.291241 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3f599-3375-46d6-a57d-7a92116ca8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:02.291696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3f599-3375-46d6-a57d-7a92116ca8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:02.292013 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6a2106b8-ca5a-4ffa-8f90-d2cea1ea792f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:02.326629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-d5a3f599-3375-46d6-a57d-7a92116ca8a9 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:02.328079 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d5a3f599-3375-46d6-a57d-7a92116ca8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:02.339327 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:02.400368 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 994/1961] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:02.406177 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2106b8-ca5a-4ffa-8f90-d2cea1ea792f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:02.419779 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2106b8-ca5a-4ffa-8f90-d2cea1ea792f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:02.424777 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97053732-13cd-4e3e-b396-d2741f3e4468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:02.425964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97053732-13cd-4e3e-b396-d2741f3e4468 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:02.426393 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97053732-13cd-4e3e-b396-d2741f3e4468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:02.426885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97053732-13cd-4e3e-b396-d2741f3e4468 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:02.427731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97053732-13cd-4e3e-b396-d2741f3e4468 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:31:02.428178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97053732-13cd-4e3e-b396-d2741f3e4468 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:02.429578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 968/1962] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:02.430440 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97053732-13cd-4e3e-b396-d2741f3e4468 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:02.442599 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3f599-3375-46d6-a57d-7a92116ca8a9 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:02.446322 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d5a3f599-3375-46d6-a57d-7a92116ca8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:02.447119 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 995/1963] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:31:02.457412 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97053732-13cd-4e3e-b396-d2741f3e4468 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:02.469742 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97053732-13cd-4e3e-b396-d2741f3e4468 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:02.476159 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820bad08-cdcf-4f93-a002-d6b36310f4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:02.476999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820bad08-cdcf-4f93-a002-d6b36310f4fd admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:02.477636 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820bad08-cdcf-4f93-a002-d6b36310f4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:02.478054 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820bad08-cdcf-4f93-a002-d6b36310f4fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:02.478890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820bad08-cdcf-4f93-a002-d6b36310f4fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:02.481115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820bad08-cdcf-4f93-a002-d6b36310f4fd admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:02.525802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 996/1964] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:31:02 2022] POST /identity/v3/projects => generated 379 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:31:02.527547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820bad08-cdcf-4f93-a002-d6b36310f4fd admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:02.535452 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3f599-3375-46d6-a57d-7a92116ca8a9 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:02.600064 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f13dc-d29b-41ed-91db-6c66650ed133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:02.600752 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f13dc-d29b-41ed-91db-6c66650ed133 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:02.601115 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f13dc-d29b-41ed-91db-6c66650ed133 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:02.601463 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f13dc-d29b-41ed-91db-6c66650ed133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:02.605377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:02.633695 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820bad08-cdcf-4f93-a002-d6b36310f4fd admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:02.641183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871c145-5334-4254-86e9-7177745e3e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:02.641828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871c145-5334-4254-86e9-7177745e3e7a admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:02.642112 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871c145-5334-4254-86e9-7177745e3e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:02.642395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871c145-5334-4254-86e9-7177745e3e7a admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:02.643328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871c145-5334-4254-86e9-7177745e3e7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:02.645549 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871c145-5334-4254-86e9-7177745e3e7a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:02.692809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 997/1965] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:31:02 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:31:02.694152 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871c145-5334-4254-86e9-7177745e3e7a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:02.700287 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e871c145-5334-4254-86e9-7177745e3e7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:02.728188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871c145-5334-4254-86e9-7177745e3e7a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:02.735909 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3f13dc-d29b-41ed-91db-6c66650ed133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:02.738130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3f13dc-d29b-41ed-91db-6c66650ed133 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:02.742541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d61845-e258-425f-9119-ddddde484beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:02.743276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d61845-e258-425f-9119-ddddde484beb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:02.744324 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d61845-e258-425f-9119-ddddde484beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:02.744618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d61845-e258-425f-9119-ddddde484beb admin admin] PATH_INFO: `/v3/projects/1e4506bcf73d4f4089467c2cc32cd801` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:02.748018 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d61845-e258-425f-9119-ddddde484beb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e4506bcf73d4f4089467c2cc32cd801)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:02.750185 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d61845-e258-425f-9119-ddddde484beb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:02.754130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 969/1966] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:31:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:31:02.755492 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f13dc-d29b-41ed-91db-6c66650ed133 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:02.767826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f13dc-d29b-41ed-91db-6c66650ed133 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:02.775625 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a2cd8-a348-457c-9058-107b520e9591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:02.777085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a2cd8-a348-457c-9058-107b520e9591 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:02.777373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a2cd8-a348-457c-9058-107b520e9591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:02.777654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a2cd8-a348-457c-9058-107b520e9591 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:02.778654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59a2cd8-a348-457c-9058-107b520e9591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:02.781086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59a2cd8-a348-457c-9058-107b520e9591 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:02.794034 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 970/1967] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:31:02.799583 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a2cd8-a348-457c-9058-107b520e9591 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:02.830891 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a2cd8-a348-457c-9058-107b520e9591 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:02.839977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:02.852727 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-45d61845-e258-425f-9119-ddddde484beb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e4506bcf73d4f4089467c2cc32cd801', 'request_id': 'req-45d61845-e258-425f-9119-ddddde484beb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:02.863492 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-45d61845-e258-425f-9119-ddddde484beb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e4506bcf73d4f4089467c2cc32cd801', 'request_id': 'req-45d61845-e258-425f-9119-ddddde484beb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:02.875394 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-45d61845-e258-425f-9119-ddddde484beb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e4506bcf73d4f4089467c2cc32cd801', 'request_id': 'req-45d61845-e258-425f-9119-ddddde484beb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:02.916004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-45d61845-e258-425f-9119-ddddde484beb admin admin] The token cache is being invalidate because project 1e4506bcf73d4f4089467c2cc32cd801 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:31:02.916354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-45d61845-e258-425f-9119-ddddde484beb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:02.921311 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc247d-ab92-4d09-a6e9-20a43e3c7f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:02.921467 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-45d61845-e258-425f-9119-ddddde484beb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:02.921574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc247d-ab92-4d09-a6e9-20a43e3c7f4f admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:02.921818 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc247d-ab92-4d09-a6e9-20a43e3c7f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:02.922165 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc247d-ab92-4d09-a6e9-20a43e3c7f4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:02.922920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 998/1968] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:31:02 2022] DELETE /identity/v3/projects/1e4506bcf73d4f4089467c2cc32cd801 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:02.927024 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc247d-ab92-4d09-a6e9-20a43e3c7f4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:02.927024 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc247d-ab92-4d09-a6e9-20a43e3c7f4f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:02.928294 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebbd84-743a-4ae7-92f1-3a4da05182e0 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:02.928605 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebbd84-743a-4ae7-92f1-3a4da05182e0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:02.928879 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebbd84-743a-4ae7-92f1-3a4da05182e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:02.950772 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 971/1969] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:31:02.955351 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc247d-ab92-4d09-a6e9-20a43e3c7f4f admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:02.955602 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-83ebbd84-743a-4ae7-92f1-3a4da05182e0 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:02.956953 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-83ebbd84-743a-4ae7-92f1-3a4da05182e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:02.958316 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c9fc247d-ab92-4d09-a6e9-20a43e3c7f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:02.966734 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 999/1970] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:31:02.967520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ebbd84-743a-4ae7-92f1-3a4da05182e0 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:02.968834 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-83ebbd84-743a-4ae7-92f1-3a4da05182e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:02.978245 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc247d-ab92-4d09-a6e9-20a43e3c7f4f admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:02.983405 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609b5d3-21cd-4386-be55-f1b35b8504e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:02.984326 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609b5d3-21cd-4386-be55-f1b35b8504e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:02.985106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609b5d3-21cd-4386-be55-f1b35b8504e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:02.985582 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609b5d3-21cd-4386-be55-f1b35b8504e1 admin admin] PATH_INFO: `/v3/projects/64256c19c24d4efc90fa0ee6651a2a6e/users/4eef477e773c4af58e471775ffaf8011/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:02.989075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ebbd84-743a-4ae7-92f1-3a4da05182e0 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:03.002144 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d686eaf-2c71-45bb-93e5-b496eee4df25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:03.002851 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d686eaf-2c71-45bb-93e5-b496eee4df25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:03.003188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d686eaf-2c71-45bb-93e5-b496eee4df25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:03.004174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d686eaf-2c71-45bb-93e5-b496eee4df25 admin admin] PATH_INFO: `/v3/projects/5ba6d5b00cc14526819421a6a3b290fc` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:03.005573 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d686eaf-2c71-45bb-93e5-b496eee4df25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ba6d5b00cc14526819421a6a3b290fc)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:03.007911 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d686eaf-2c71-45bb-93e5-b496eee4df25 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:03.031742 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d609b5d3-21cd-4386-be55-f1b35b8504e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64256c19c24d4efc90fa0ee6651a2a6e, user_id=4eef477e773c4af58e471775ffaf8011, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:03.035365 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d609b5d3-21cd-4386-be55-f1b35b8504e1 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:03.066079 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 972/1971] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:31:02 2022] PUT /identity/v3/projects/64256c19c24d4efc90fa0ee6651a2a6e/users/4eef477e773c4af58e471775ffaf8011/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:03.068163 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b267da-ca4c-4d2d-ae25-c3bba1354331 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:03.068718 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b267da-ca4c-4d2d-ae25-c3bba1354331 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:03.068828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b267da-ca4c-4d2d-ae25-c3bba1354331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:03.088549 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0d686eaf-2c71-45bb-93e5-b496eee4df25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ba6d5b00cc14526819421a6a3b290fc', 'request_id': 'req-0d686eaf-2c71-45bb-93e5-b496eee4df25'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.096051 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0d686eaf-2c71-45bb-93e5-b496eee4df25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ba6d5b00cc14526819421a6a3b290fc', 'request_id': 'req-0d686eaf-2c71-45bb-93e5-b496eee4df25'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.103312 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-d1b267da-ca4c-4d2d-ae25-c3bba1354331 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:03.104972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d1b267da-ca4c-4d2d-ae25-c3bba1354331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:03.107648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0d686eaf-2c71-45bb-93e5-b496eee4df25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ba6d5b00cc14526819421a6a3b290fc', 'request_id': 'req-0d686eaf-2c71-45bb-93e5-b496eee4df25'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.124280 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0d686eaf-2c71-45bb-93e5-b496eee4df25 admin admin] The token cache is being invalidate because project 5ba6d5b00cc14526819421a6a3b290fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:31:03.124530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0d686eaf-2c71-45bb-93e5-b496eee4df25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.126368 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0d686eaf-2c71-45bb-93e5-b496eee4df25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.129266 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1000/1972] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:31:02 2022] DELETE /identity/v3/projects/5ba6d5b00cc14526819421a6a3b290fc => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:03.132463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c9d52-53b2-47c4-90d6-5d997f3d3aa7 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:03.132757 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c9d52-53b2-47c4-90d6-5d997f3d3aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:03.133036 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c9d52-53b2-47c4-90d6-5d997f3d3aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:03.162348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5e2c9d52-53b2-47c4-90d6-5d997f3d3aa7 None None] MFA Rules not processed for user `4eef477e773c4af58e471775ffaf8011`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:03.163586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5e2c9d52-53b2-47c4-90d6-5d997f3d3aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:03.187891 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 973/1973] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:03.188943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b267da-ca4c-4d2d-ae25-c3bba1354331 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:03.190386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d1b267da-ca4c-4d2d-ae25-c3bba1354331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:03.197507 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=81,keystone:INSERT=5,keystone:DELETE=14 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:03.209246 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b267da-ca4c-4d2d-ae25-c3bba1354331 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:03.214283 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1043d-3db2-430a-b172-d24da58d3cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:03.214935 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1043d-3db2-430a-b172-d24da58d3cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:03.215225 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1043d-3db2-430a-b172-d24da58d3cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:03.215506 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1043d-3db2-430a-b172-d24da58d3cbf admin admin] PATH_INFO: `/v3/users/0efcd370bf9b4a51b746b95a88a2b0ac` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:03.217100 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f1043d-3db2-430a-b172-d24da58d3cbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=0efcd370bf9b4a51b746b95a88a2b0ac)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:03.219092 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f1043d-3db2-430a-b172-d24da58d3cbf admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:03.246795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1001/1974] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:03.247705 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c9d52-53b2-47c4-90d6-5d997f3d3aa7 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:03.262946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c9d52-53b2-47c4-90d6-5d997f3d3aa7 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:03.269445 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9274cc-2da5-4bc6-a1d2-d225c95f7e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:03.270081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9274cc-2da5-4bc6-a1d2-d225c95f7e0b admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:03.270364 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9274cc-2da5-4bc6-a1d2-d225c95f7e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:03.270642 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9274cc-2da5-4bc6-a1d2-d225c95f7e0b admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:03.271417 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9274cc-2da5-4bc6-a1d2-d225c95f7e0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:31:03.271646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9274cc-2da5-4bc6-a1d2-d225c95f7e0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:03.273285 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9274cc-2da5-4bc6-a1d2-d225c95f7e0b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:03.284961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-40f1043d-3db2-430a-b172-d24da58d3cbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0efcd370bf9b4a51b746b95a88a2b0ac'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.288511 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1002/1975] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:31:03.288646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9274cc-2da5-4bc6-a1d2-d225c95f7e0b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:03.291941 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1a9274cc-2da5-4bc6-a1d2-d225c95f7e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:03.293402 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-40f1043d-3db2-430a-b172-d24da58d3cbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0efcd370bf9b4a51b746b95a88a2b0ac'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.307011 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-40f1043d-3db2-430a-b172-d24da58d3cbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0efcd370bf9b4a51b746b95a88a2b0ac'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.307955 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9274cc-2da5-4bc6-a1d2-d225c95f7e0b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:03.308069 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-40f1043d-3db2-430a-b172-d24da58d3cbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0efcd370bf9b4a51b746b95a88a2b0ac'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.308638 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-40f1043d-3db2-430a-b172-d24da58d3cbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0efcd370bf9b4a51b746b95a88a2b0ac'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.311966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:03.317418 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af1308-162f-4507-a43e-dbb072513517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:03.319956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af1308-162f-4507-a43e-dbb072513517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:03.321050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af1308-162f-4507-a43e-dbb072513517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:03.322586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af1308-162f-4507-a43e-dbb072513517 admin admin] PATH_INFO: `/v3/projects/ac509f3bfdfa49149c4cc3f5d63d0932` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:03.325792 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af1308-162f-4507-a43e-dbb072513517 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac509f3bfdfa49149c4cc3f5d63d0932)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:03.328112 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af1308-162f-4507-a43e-dbb072513517 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:03.333230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 974/1976] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:31:03 2022] DELETE /identity/v3/users/0efcd370bf9b4a51b746b95a88a2b0ac => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:03.333498 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1043d-3db2-430a-b172-d24da58d3cbf admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:03.349035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-40f1043d-3db2-430a-b172-d24da58d3cbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:03.434837 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-41af1308-162f-4507-a43e-dbb072513517 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac509f3bfdfa49149c4cc3f5d63d0932', 'request_id': 'req-41af1308-162f-4507-a43e-dbb072513517'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.441764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-41af1308-162f-4507-a43e-dbb072513517 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac509f3bfdfa49149c4cc3f5d63d0932', 'request_id': 'req-41af1308-162f-4507-a43e-dbb072513517'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.461738 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-41af1308-162f-4507-a43e-dbb072513517 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac509f3bfdfa49149c4cc3f5d63d0932', 'request_id': 'req-41af1308-162f-4507-a43e-dbb072513517'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.466764 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1043d-3db2-430a-b172-d24da58d3cbf admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:03.480901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-41af1308-162f-4507-a43e-dbb072513517 admin admin] The token cache is being invalidate because project ac509f3bfdfa49149c4cc3f5d63d0932 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:31:03.481725 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-41af1308-162f-4507-a43e-dbb072513517 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.481890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-41af1308-162f-4507-a43e-dbb072513517 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.483601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1003/1977] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:31:03 2022] DELETE /identity/v3/projects/ac509f3bfdfa49149c4cc3f5d63d0932 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:03.484225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af1308-162f-4507-a43e-dbb072513517 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:03.485509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-41af1308-162f-4507-a43e-dbb072513517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:03.544386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5171a0-94fa-4570-97b0-282924290e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:03.545189 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5171a0-94fa-4570-97b0-282924290e6d admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:03.545510 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5171a0-94fa-4570-97b0-282924290e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:03.546051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5171a0-94fa-4570-97b0-282924290e6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:03.546825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5171a0-94fa-4570-97b0-282924290e6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:03.548899 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5171a0-94fa-4570-97b0-282924290e6d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:03.567393 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af1308-162f-4507-a43e-dbb072513517 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:03.575244 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012ba26-4c3b-4696-b486-ca5877f7244a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:03.575878 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012ba26-4c3b-4696-b486-ca5877f7244a admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:03.576164 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012ba26-4c3b-4696-b486-ca5877f7244a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:03.576505 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012ba26-4c3b-4696-b486-ca5877f7244a admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:03.577184 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012ba26-4c3b-4696-b486-ca5877f7244a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:03.579116 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012ba26-4c3b-4696-b486-ca5877f7244a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:03.584153 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 975/1978] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:31:03 2022] POST /identity/v3/projects => generated 379 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:31:03.584153 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5171a0-94fa-4570-97b0-282924290e6d admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:03.588474 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ca5171a0-94fa-4570-97b0-282924290e6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:03.617919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1004/1979] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:31:03 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:31:03.621476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012ba26-4c3b-4696-b486-ca5877f7244a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:03.625972 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7012ba26-4c3b-4696-b486-ca5877f7244a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:03.662974 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:03.692735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5171a0-94fa-4570-97b0-282924290e6d admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:03.726136 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012ba26-4c3b-4696-b486-ca5877f7244a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:03.731023 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c6d3d-cba2-4749-9ce6-c5e2c90f57c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:03.731651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c6d3d-cba2-4749-9ce6-c5e2c90f57c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:03.731935 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c6d3d-cba2-4749-9ce6-c5e2c90f57c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:03.732211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c6d3d-cba2-4749-9ce6-c5e2c90f57c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:03.732944 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c6d3d-cba2-4749-9ce6-c5e2c90f57c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:03.734868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c6d3d-cba2-4749-9ce6-c5e2c90f57c2 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:03.738154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:03.759492 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1005/1980] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:31:03 2022] POST /identity/v3/users => generated 296 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:31:03.760646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c6d3d-cba2-4749-9ce6-c5e2c90f57c2 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:03.766190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c576f-626b-45e7-b848-78675d336d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:03.767720 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c576f-626b-45e7-b848-78675d336d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:03.768236 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c576f-626b-45e7-b848-78675d336d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:03.769291 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c576f-626b-45e7-b848-78675d336d1b admin admin] PATH_INFO: `/v3/users/186500f6f7bb4836a2c25192a5c4102f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:03.771015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c576f-626b-45e7-b848-78675d336d1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=186500f6f7bb4836a2c25192a5c4102f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:03.773029 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c576f-626b-45e7-b848-78675d336d1b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:03.773223 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c6d3d-cba2-4749-9ce6-c5e2c90f57c2 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:03.778606 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9fde9-617f-4b25-a7a0-818d8a40fbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:03.779233 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9fde9-617f-4b25-a7a0-818d8a40fbdb admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:03.779513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9fde9-617f-4b25-a7a0-818d8a40fbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:03.780012 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9fde9-617f-4b25-a7a0-818d8a40fbdb admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:03.780709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e9fde9-617f-4b25-a7a0-818d8a40fbdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:03.782966 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e9fde9-617f-4b25-a7a0-818d8a40fbdb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:03.823906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1006/1981] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:31:03 2022] POST /identity/v3/users => generated 287 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:31:03.824823 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9fde9-617f-4b25-a7a0-818d8a40fbdb admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:03.826458 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-24e9fde9-617f-4b25-a7a0-818d8a40fbdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:03.833337 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fd8c576f-626b-45e7-b848-78675d336d1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '186500f6f7bb4836a2c25192a5c4102f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.840126 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9fde9-617f-4b25-a7a0-818d8a40fbdb admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:03.840566 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fd8c576f-626b-45e7-b848-78675d336d1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '186500f6f7bb4836a2c25192a5c4102f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.845071 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85012ba7-d639-4fc6-8cf7-abc7f9537f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:03.845830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85012ba7-d639-4fc6-8cf7-abc7f9537f23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:03.846208 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85012ba7-d639-4fc6-8cf7-abc7f9537f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:03.846558 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85012ba7-d639-4fc6-8cf7-abc7f9537f23 admin admin] PATH_INFO: `/v3/projects/703700e39c414c5b87be07aa8d07bf80` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:03.848577 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85012ba7-d639-4fc6-8cf7-abc7f9537f23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=703700e39c414c5b87be07aa8d07bf80)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:03.850724 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85012ba7-d639-4fc6-8cf7-abc7f9537f23 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:03.855437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fd8c576f-626b-45e7-b848-78675d336d1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '186500f6f7bb4836a2c25192a5c4102f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.856164 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fd8c576f-626b-45e7-b848-78675d336d1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '186500f6f7bb4836a2c25192a5c4102f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.856852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fd8c576f-626b-45e7-b848-78675d336d1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '186500f6f7bb4836a2c25192a5c4102f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.874735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 976/1982] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:31:03 2022] DELETE /identity/v3/users/186500f6f7bb4836a2c25192a5c4102f => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:03.878632 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c576f-626b-45e7-b848-78675d336d1b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:03.880039 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fd8c576f-626b-45e7-b848-78675d336d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:03.932268 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-85012ba7-d639-4fc6-8cf7-abc7f9537f23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '703700e39c414c5b87be07aa8d07bf80', 'request_id': 'req-85012ba7-d639-4fc6-8cf7-abc7f9537f23'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.938827 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-85012ba7-d639-4fc6-8cf7-abc7f9537f23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '703700e39c414c5b87be07aa8d07bf80', 'request_id': 'req-85012ba7-d639-4fc6-8cf7-abc7f9537f23'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.948205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-85012ba7-d639-4fc6-8cf7-abc7f9537f23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '703700e39c414c5b87be07aa8d07bf80', 'request_id': 'req-85012ba7-d639-4fc6-8cf7-abc7f9537f23'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.955887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c576f-626b-45e7-b848-78675d336d1b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:03.967170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-85012ba7-d639-4fc6-8cf7-abc7f9537f23 admin admin] The token cache is being invalidate because project 703700e39c414c5b87be07aa8d07bf80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:31:03.967478 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-85012ba7-d639-4fc6-8cf7-abc7f9537f23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.968251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-85012ba7-d639-4fc6-8cf7-abc7f9537f23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:03.970046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1007/1983] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:31:03 2022] DELETE /identity/v3/projects/703700e39c414c5b87be07aa8d07bf80 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:03.974435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85012ba7-d639-4fc6-8cf7-abc7f9537f23 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:03.975969 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-85012ba7-d639-4fc6-8cf7-abc7f9537f23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:04.030533 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5116584-d0ab-4faa-9a4c-0a17cf60e5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:04.031176 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5116584-d0ab-4faa-9a4c-0a17cf60e5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:04.031493 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5116584-d0ab-4faa-9a4c-0a17cf60e5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:04.031877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5116584-d0ab-4faa-9a4c-0a17cf60e5d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:04.032681 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5116584-d0ab-4faa-9a4c-0a17cf60e5d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:04.034986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5116584-d0ab-4faa-9a4c-0a17cf60e5d1 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:04.036350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:04.047275 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 977/1984] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:31:04.053901 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ed1b7-5a68-41f4-8346-ec1648126d66 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:04.053901 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ed1b7-5a68-41f4-8346-ec1648126d66 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:04.053901 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ed1b7-5a68-41f4-8346-ec1648126d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:04.054385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85012ba7-d639-4fc6-8cf7-abc7f9537f23 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:04.059656 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc21272-7c0d-4dac-b8af-9b9104fcc794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:04.060649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc21272-7c0d-4dac-b8af-9b9104fcc794 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:04.060956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc21272-7c0d-4dac-b8af-9b9104fcc794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:04.061215 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc21272-7c0d-4dac-b8af-9b9104fcc794 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:04.062893 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc21272-7c0d-4dac-b8af-9b9104fcc794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:04.064647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc21272-7c0d-4dac-b8af-9b9104fcc794 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:04.074492 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-276ed1b7-5a68-41f4-8346-ec1648126d66 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:04.075715 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-276ed1b7-5a68-41f4-8346-ec1648126d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:04.077491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1008/1985] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:31:04.078577 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc21272-7c0d-4dac-b8af-9b9104fcc794 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:04.080578 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0cc21272-7c0d-4dac-b8af-9b9104fcc794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:04.084467 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 978/1986] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:04.085133 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ed1b7-5a68-41f4-8346-ec1648126d66 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:04.095340 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ed1b7-5a68-41f4-8346-ec1648126d66 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:04.098080 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc21272-7c0d-4dac-b8af-9b9104fcc794 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:04.101117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d913c2c-b20e-4089-81ba-399b7adca83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:04.101766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d913c2c-b20e-4089-81ba-399b7adca83c admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:04.102047 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d913c2c-b20e-4089-81ba-399b7adca83c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:04.102319 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d913c2c-b20e-4089-81ba-399b7adca83c admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:04.103058 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d913c2c-b20e-4089-81ba-399b7adca83c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:04.103403 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9daa790-aed2-4e44-8b5f-6fa552dfb26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:04.104196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9daa790-aed2-4e44-8b5f-6fa552dfb26a admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:04.104580 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9daa790-aed2-4e44-8b5f-6fa552dfb26a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:04.104834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d913c2c-b20e-4089-81ba-399b7adca83c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:04.105190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9daa790-aed2-4e44-8b5f-6fa552dfb26a admin admin] PATH_INFO: `/v3/projects/826d67a82a934deca82a6556c703a644/users/4f31ce6ec71942618f9f51e45ba0f9ec/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:04.119193 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 979/1987] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:31:04.120506 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1765443-7315-4ac0-ad33-72c22537e9fa None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:04.120962 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1765443-7315-4ac0-ad33-72c22537e9fa None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:04.121277 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1765443-7315-4ac0-ad33-72c22537e9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:04.129100 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:04.135640 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9daa790-aed2-4e44-8b5f-6fa552dfb26a admin admin] RBAC: Authorizing `identity:create_grant(project_id=826d67a82a934deca82a6556c703a644, user_id=4f31ce6ec71942618f9f51e45ba0f9ec, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:04.138744 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9daa790-aed2-4e44-8b5f-6fa552dfb26a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:04.143930 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-f1765443-7315-4ac0-ad33-72c22537e9fa None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:04.145544 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f1765443-7315-4ac0-ad33-72c22537e9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:04.155681 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 980/1988] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:04.156256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1009/1989] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:31:04 2022] PUT /identity/v3/projects/826d67a82a934deca82a6556c703a644/users/4f31ce6ec71942618f9f51e45ba0f9ec/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:04.157019 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1765443-7315-4ac0-ad33-72c22537e9fa None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:04.160258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9daa790-aed2-4e44-8b5f-6fa552dfb26a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:04.167467 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1765443-7315-4ac0-ad33-72c22537e9fa None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:04.235118 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9daa790-aed2-4e44-8b5f-6fa552dfb26a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:04.236428 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd26ea8-caaa-43a2-b0bf-aba889b0d46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:04.237129 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd26ea8-caaa-43a2-b0bf-aba889b0d46c admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:04.237431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd26ea8-caaa-43a2-b0bf-aba889b0d46c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:04.237775 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd26ea8-caaa-43a2-b0bf-aba889b0d46c admin admin] PATH_INFO: `/v3/projects/c31ed4d577414dd0be841b28e1795bae/users/6e424addc3e3403991b16ca321239d00/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:04.242464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b16d8e-eaf2-4c05-abde-38b7def6198a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:04.242464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b16d8e-eaf2-4c05-abde-38b7def6198a admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:04.242464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b16d8e-eaf2-4c05-abde-38b7def6198a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:04.242464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b16d8e-eaf2-4c05-abde-38b7def6198a admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:04.245079 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b16d8e-eaf2-4c05-abde-38b7def6198a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:31:04.245552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b16d8e-eaf2-4c05-abde-38b7def6198a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:04.247420 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b16d8e-eaf2-4c05-abde-38b7def6198a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:04.255842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd26ea8-caaa-43a2-b0bf-aba889b0d46c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c31ed4d577414dd0be841b28e1795bae, user_id=6e424addc3e3403991b16ca321239d00, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:04.258893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd26ea8-caaa-43a2-b0bf-aba889b0d46c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:04.262159 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1010/1990] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:31:04.263163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b16d8e-eaf2-4c05-abde-38b7def6198a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:04.275473 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b16d8e-eaf2-4c05-abde-38b7def6198a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:04.285729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 981/1991] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:31:04 2022] PUT /identity/v3/projects/c31ed4d577414dd0be841b28e1795bae/users/6e424addc3e3403991b16ca321239d00/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:04.285729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd26ea8-caaa-43a2-b0bf-aba889b0d46c admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:04.369716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235b92d-31e3-476e-973a-3cc3229e757e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:04.370340 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235b92d-31e3-476e-973a-3cc3229e757e admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:04.370617 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235b92d-31e3-476e-973a-3cc3229e757e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:04.371195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235b92d-31e3-476e-973a-3cc3229e757e admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:04.371589 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3235b92d-31e3-476e-973a-3cc3229e757e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:04.373604 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3235b92d-31e3-476e-973a-3cc3229e757e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:04.375099 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd26ea8-caaa-43a2-b0bf-aba889b0d46c admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:04.380834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be9ac1-d783-468e-8e35-1ae78a7e0b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:04.381555 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be9ac1-d783-468e-8e35-1ae78a7e0b6c admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:04.381932 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be9ac1-d783-468e-8e35-1ae78a7e0b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:04.382272 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be9ac1-d783-468e-8e35-1ae78a7e0b6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:04.382998 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be9ac1-d783-468e-8e35-1ae78a7e0b6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:04.383598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1011/1992] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:31:04.384964 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be9ac1-d783-468e-8e35-1ae78a7e0b6c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:04.385080 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b166e-1b67-408e-b482-5a95cccd96c1 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:04.385287 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b166e-1b67-408e-b482-5a95cccd96c1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:04.385564 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b166e-1b67-408e-b482-5a95cccd96c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:04.413403 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 982/1993] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:31:04 2022] POST /identity/v3/projects => generated 381 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:31:04.414265 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be9ac1-d783-468e-8e35-1ae78a7e0b6c admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:04.414937 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-474b166e-1b67-408e-b482-5a95cccd96c1 None None] MFA Rules not processed for user `6e424addc3e3403991b16ca321239d00`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:04.415856 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-70be9ac1-d783-468e-8e35-1ae78a7e0b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:04.416578 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-474b166e-1b67-408e-b482-5a95cccd96c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:04.485658 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:04.512825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1012/1994] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:04.513576 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b166e-1b67-408e-b482-5a95cccd96c1 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:04.521850 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be9ac1-d783-468e-8e35-1ae78a7e0b6c admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:04.527961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1342f54-9fb9-410a-97b4-0422167851ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:04.528659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1342f54-9fb9-410a-97b4-0422167851ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:04.529135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1342f54-9fb9-410a-97b4-0422167851ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:04.529440 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1342f54-9fb9-410a-97b4-0422167851ff admin admin] PATH_INFO: `/v3/projects/05076cfafedc42a792b60a3de8775728` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:04.529689 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b166e-1b67-408e-b482-5a95cccd96c1 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:04.530725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1342f54-9fb9-410a-97b4-0422167851ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=05076cfafedc42a792b60a3de8775728)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:04.532831 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1342f54-9fb9-410a-97b4-0422167851ff admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:04.537880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec485be-8925-49d0-a58a-6e28c19beb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:04.538673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec485be-8925-49d0-a58a-6e28c19beb49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:04.539358 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec485be-8925-49d0-a58a-6e28c19beb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:04.539820 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec485be-8925-49d0-a58a-6e28c19beb49 admin admin] PATH_INFO: `/v3/projects/826d67a82a934deca82a6556c703a644/users/4f31ce6ec71942618f9f51e45ba0f9ec/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:04.541982 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec485be-8925-49d0-a58a-6e28c19beb49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=826d67a82a934deca82a6556c703a644, user_id=4f31ce6ec71942618f9f51e45ba0f9ec, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:04.545077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec485be-8925-49d0-a58a-6e28c19beb49 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:04.552897 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:04.571194 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1013/1995] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:31:04 2022] PUT /identity/v3/projects/826d67a82a934deca82a6556c703a644/users/4f31ce6ec71942618f9f51e45ba0f9ec/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:04.571194 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec485be-8925-49d0-a58a-6e28c19beb49 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:04.608158 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a1342f54-9fb9-410a-97b4-0422167851ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05076cfafedc42a792b60a3de8775728', 'request_id': 'req-a1342f54-9fb9-410a-97b4-0422167851ff'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:04.618891 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a1342f54-9fb9-410a-97b4-0422167851ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05076cfafedc42a792b60a3de8775728', 'request_id': 'req-a1342f54-9fb9-410a-97b4-0422167851ff'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:04.626900 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a1342f54-9fb9-410a-97b4-0422167851ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05076cfafedc42a792b60a3de8775728', 'request_id': 'req-a1342f54-9fb9-410a-97b4-0422167851ff'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:04.640900 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a1342f54-9fb9-410a-97b4-0422167851ff admin admin] The token cache is being invalidate because project 05076cfafedc42a792b60a3de8775728 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:31:04.641538 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a1342f54-9fb9-410a-97b4-0422167851ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:04.642385 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a1342f54-9fb9-410a-97b4-0422167851ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:04.643024 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec485be-8925-49d0-a58a-6e28c19beb49 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:04.644928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 983/1996] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:31:04 2022] DELETE /identity/v3/projects/05076cfafedc42a792b60a3de8775728 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:04.645712 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1342f54-9fb9-410a-97b4-0422167851ff admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:04.647185 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a1342f54-9fb9-410a-97b4-0422167851ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:04.710198 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e08da-fc1e-4654-9d1c-be7441c117f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:04.710904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e08da-fc1e-4654-9d1c-be7441c117f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:04.711117 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e08da-fc1e-4654-9d1c-be7441c117f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:04.711388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e08da-fc1e-4654-9d1c-be7441c117f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:04.712110 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e08da-fc1e-4654-9d1c-be7441c117f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:04.714066 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e08da-fc1e-4654-9d1c-be7441c117f3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:04.716443 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1342f54-9fb9-410a-97b4-0422167851ff admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:04.721698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b3f9c-6ef7-445c-aee6-f420e9d26306 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:04.723347 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b3f9c-6ef7-445c-aee6-f420e9d26306 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:04.723748 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b3f9c-6ef7-445c-aee6-f420e9d26306 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:04.724333 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b3f9c-6ef7-445c-aee6-f420e9d26306 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:04.726374 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-313b3f9c-6ef7-445c-aee6-f420e9d26306 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:04.739787 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1014/1997] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:31:04 2022] POST /identity/v3/users => generated 291 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:31:04.741514 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f69da-62f7-4881-8315-b130bb9f8820 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:04.741874 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f69da-62f7-4881-8315-b130bb9f8820 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:04.742226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f69da-62f7-4881-8315-b130bb9f8820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:04.769998 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-fa4f69da-62f7-4881-8315-b130bb9f8820 None None] MFA Rules not processed for user `4f31ce6ec71942618f9f51e45ba0f9ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:04.771251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fa4f69da-62f7-4881-8315-b130bb9f8820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:04.793865 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b3f9c-6ef7-445c-aee6-f420e9d26306 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:04.795373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b3f9c-6ef7-445c-aee6-f420e9d26306 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:04.807680 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 984/1998] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:31:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:31:04.809014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a1008-b355-4a5e-b545-dd843015f751 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:04.809343 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a1008-b355-4a5e-b545-dd843015f751 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:04.809762 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a1008-b355-4a5e-b545-dd843015f751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:04.833407 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ae8a1008-b355-4a5e-b545-dd843015f751 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:04.834711 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ae8a1008-b355-4a5e-b545-dd843015f751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:04.843846 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 985/1999] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:04.852699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a1008-b355-4a5e-b545-dd843015f751 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:04.855050 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ae8a1008-b355-4a5e-b545-dd843015f751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:04.867006 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1015/2000] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:04 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:04.868290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f7c66-454c-4e53-b972-109e60e336b0 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:04.868603 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f7c66-454c-4e53-b972-109e60e336b0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:04.868926 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f7c66-454c-4e53-b972-109e60e336b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:04.870136 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a1008-b355-4a5e-b545-dd843015f751 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:04.889557 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf4357-842a-47a1-8cfb-56e2985aa903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:04.890008 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf4357-842a-47a1-8cfb-56e2985aa903 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:04.890283 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf4357-842a-47a1-8cfb-56e2985aa903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:04.890575 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf4357-842a-47a1-8cfb-56e2985aa903 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:04.891294 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdf4357-842a-47a1-8cfb-56e2985aa903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:04.892930 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdf4357-842a-47a1-8cfb-56e2985aa903 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:04.907022 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 986/2001] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:31:04.907022 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf4357-842a-47a1-8cfb-56e2985aa903 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:04.908168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f12f7c66-454c-4e53-b972-109e60e336b0 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:04.910063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f12f7c66-454c-4e53-b972-109e60e336b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:04.910215 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7bdf4357-842a-47a1-8cfb-56e2985aa903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:04.924540 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf4357-842a-47a1-8cfb-56e2985aa903 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:04.928193 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1016/2002] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:04.929650 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f7c66-454c-4e53-b972-109e60e336b0 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:04.931536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61f043-6356-4b40-a5c7-a6654cc8240f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:04.932177 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61f043-6356-4b40-a5c7-a6654cc8240f service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:04.932544 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61f043-6356-4b40-a5c7-a6654cc8240f service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:04.932820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61f043-6356-4b40-a5c7-a6654cc8240f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:04.940143 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:04.946500 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61f043-6356-4b40-a5c7-a6654cc8240f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:04.946693 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f7c66-454c-4e53-b972-109e60e336b0 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:04.949232 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61f043-6356-4b40-a5c7-a6654cc8240f service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:04.953127 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bead2e8-6283-40d9-847b-026bac65681d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:04.954078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bead2e8-6283-40d9-847b-026bac65681d admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:04.954364 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bead2e8-6283-40d9-847b-026bac65681d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:04.954785 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bead2e8-6283-40d9-847b-026bac65681d admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:04.955493 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bead2e8-6283-40d9-847b-026bac65681d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:04.959897 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bead2e8-6283-40d9-847b-026bac65681d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:04.974795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1017/2003] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:31:04.975536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bead2e8-6283-40d9-847b-026bac65681d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:04.979231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 987/2004] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:31:04 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:31:04.980997 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:04.981745 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61f043-6356-4b40-a5c7-a6654cc8240f service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:04.988378 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bead2e8-6283-40d9-847b-026bac65681d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:05.007874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61f043-6356-4b40-a5c7-a6654cc8240f service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:05.015718 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a988b-2ada-4280-8c2b-0164e6d3a434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:05.018543 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a988b-2ada-4280-8c2b-0164e6d3a434 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:05.018727 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245a6b8-fb05-4e84-9b65-747ef9a54265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:05.019969 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a988b-2ada-4280-8c2b-0164e6d3a434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:05.020375 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245a6b8-fb05-4e84-9b65-747ef9a54265 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:05.020617 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245a6b8-fb05-4e84-9b65-747ef9a54265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:05.021041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245a6b8-fb05-4e84-9b65-747ef9a54265 admin admin] PATH_INFO: `/v3/projects/d74662f7d3bd41e7ac835755d212e877/users/7823df24b0d64b0d93a6ee0e684ed483/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:05.021595 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a988b-2ada-4280-8c2b-0164e6d3a434 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:05.022703 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a988b-2ada-4280-8c2b-0164e6d3a434 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:31:05.023370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a988b-2ada-4280-8c2b-0164e6d3a434 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:05.026448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a988b-2ada-4280-8c2b-0164e6d3a434 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:05.041118 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8245a6b8-fb05-4e84-9b65-747ef9a54265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d74662f7d3bd41e7ac835755d212e877, user_id=7823df24b0d64b0d93a6ee0e684ed483, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:05.043353 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1018/2005] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:31:05.048484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a988b-2ada-4280-8c2b-0164e6d3a434 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:05.048605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8245a6b8-fb05-4e84-9b65-747ef9a54265 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:05.065778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a988b-2ada-4280-8c2b-0164e6d3a434 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:05.070783 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 988/2006] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:31:04 2022] PUT /identity/v3/projects/d74662f7d3bd41e7ac835755d212e877/users/7823df24b0d64b0d93a6ee0e684ed483/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:05.083526 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f719d-4aeb-418a-a6e8-99d2999ef368 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:05.083843 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f719d-4aeb-418a-a6e8-99d2999ef368 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:05.084073 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f719d-4aeb-418a-a6e8-99d2999ef368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:05.125430 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-b48f719d-4aeb-418a-a6e8-99d2999ef368 None None] MFA Rules not processed for user `7823df24b0d64b0d93a6ee0e684ed483`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:05.126798 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b48f719d-4aeb-418a-a6e8-99d2999ef368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:05.173349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb7197-7819-4108-9711-f521730fc3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:05.177327 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb7197-7819-4108-9711-f521730fc3c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:05.177622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb7197-7819-4108-9711-f521730fc3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:05.177924 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb7197-7819-4108-9711-f521730fc3c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:05.178768 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb7197-7819-4108-9711-f521730fc3c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:05.180760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb7197-7819-4108-9711-f521730fc3c6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:05.218238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1019/2007] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:31:05 2022] POST /identity/v3/projects => generated 377 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:31:05.222519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb7197-7819-4108-9711-f521730fc3c6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:05.257366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 989/2008] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:05.259096 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f719d-4aeb-418a-a6e8-99d2999ef368 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:05.281863 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f719d-4aeb-418a-a6e8-99d2999ef368 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:05.314922 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb7197-7819-4108-9711-f521730fc3c6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:05.333811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e05a3e-1c45-469f-95fe-6558bd27a319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:05.334819 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e05a3e-1c45-469f-95fe-6558bd27a319 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:05.335163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e05a3e-1c45-469f-95fe-6558bd27a319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:05.335507 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e05a3e-1c45-469f-95fe-6558bd27a319 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:05.337205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e05a3e-1c45-469f-95fe-6558bd27a319 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:05.341654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e05a3e-1c45-469f-95fe-6558bd27a319 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:05.371595 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902a128-821c-416c-8c4a-653a023d18af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:05.372785 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902a128-821c-416c-8c4a-653a023d18af admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:05.373168 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902a128-821c-416c-8c4a-653a023d18af admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:05.374718 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902a128-821c-416c-8c4a-653a023d18af admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:05.375703 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902a128-821c-416c-8c4a-653a023d18af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:05.380524 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902a128-821c-416c-8c4a-653a023d18af admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:05.410023 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1020/2009] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:31:05 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:31:05.414759 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e05a3e-1c45-469f-95fe-6558bd27a319 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:05.428464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e05a3e-1c45-469f-95fe-6558bd27a319 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:05.434579 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93eaf1-2bef-494c-aaa0-1d6c5859760d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:05.435321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93eaf1-2bef-494c-aaa0-1d6c5859760d admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:05.435677 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93eaf1-2bef-494c-aaa0-1d6c5859760d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:05.436016 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93eaf1-2bef-494c-aaa0-1d6c5859760d admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:05.436865 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93eaf1-2bef-494c-aaa0-1d6c5859760d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:05.438691 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93eaf1-2bef-494c-aaa0-1d6c5859760d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:05.455867 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 990/2010] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:31:05 2022] POST /identity/v3/projects => generated 379 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:31:05.457570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902a128-821c-416c-8c4a-653a023d18af admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:05.458447 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1021/2011] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 12:31:05.459653 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93eaf1-2bef-494c-aaa0-1d6c5859760d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:05.489499 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93eaf1-2bef-494c-aaa0-1d6c5859760d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:05.535284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:05.569887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902a128-821c-416c-8c4a-653a023d18af admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:05.575787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e994f-91a4-40b4-81a9-73c4c2c226e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:05.576329 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e994f-91a4-40b4-81a9-73c4c2c226e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:05.576614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e994f-91a4-40b4-81a9-73c4c2c226e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:05.576885 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e994f-91a4-40b4-81a9-73c4c2c226e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:05.578379 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e994f-91a4-40b4-81a9-73c4c2c226e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:05.580776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e994f-91a4-40b4-81a9-73c4c2c226e1 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:05.597381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:05.607200 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 991/2012] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:31:05 2022] POST /identity/v3/users => generated 296 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:31:05.612696 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bedbc65-861e-4571-bae1-5f86c79b590e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:05.612901 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e994f-91a4-40b4-81a9-73c4c2c226e1 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:05.617231 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedbc65-861e-4571-bae1-5f86c79b590e admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:05.618314 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedbc65-861e-4571-bae1-5f86c79b590e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:05.620135 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedbc65-861e-4571-bae1-5f86c79b590e admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:05.622772 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bedbc65-861e-4571-bae1-5f86c79b590e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:05.625091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e994f-91a4-40b4-81a9-73c4c2c226e1 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:05.630674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1da828-6ced-4d12-92be-c83a635ccd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:05.631521 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1da828-6ced-4d12-92be-c83a635ccd04 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:05.631776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1da828-6ced-4d12-92be-c83a635ccd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:05.632113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1da828-6ced-4d12-92be-c83a635ccd04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:05.632502 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bedbc65-861e-4571-bae1-5f86c79b590e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:05.634453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1da828-6ced-4d12-92be-c83a635ccd04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:05.636853 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1da828-6ced-4d12-92be-c83a635ccd04 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:05.645555 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1022/2013] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:31:05.649536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bedbc65-861e-4571-bae1-5f86c79b590e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:05.653506 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 992/2014] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 12:31:05.654370 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1da828-6ced-4d12-92be-c83a635ccd04 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:05.666023 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1da828-6ced-4d12-92be-c83a635ccd04 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:05.671163 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26883c-25df-4f73-82f7-f81eba9db204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:05.671879 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26883c-25df-4f73-82f7-f81eba9db204 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:05.672165 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26883c-25df-4f73-82f7-f81eba9db204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:05.672478 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26883c-25df-4f73-82f7-f81eba9db204 admin admin] PATH_INFO: `/v3/projects/cdd0f4290eaf45c981acb531b818b55a/users/576ffe2ec3734f6fa202ef4eca81adad/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:05.691687 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26883c-25df-4f73-82f7-f81eba9db204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdd0f4290eaf45c981acb531b818b55a, user_id=576ffe2ec3734f6fa202ef4eca81adad, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:05.694441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26883c-25df-4f73-82f7-f81eba9db204 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:05.717897 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 993/2015] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:31:05 2022] PUT /identity/v3/projects/cdd0f4290eaf45c981acb531b818b55a/users/576ffe2ec3734f6fa202ef4eca81adad/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:05.717897 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26883c-25df-4f73-82f7-f81eba9db204 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:05.746248 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bedbc65-861e-4571-bae1-5f86c79b590e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:05.792177 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26883c-25df-4f73-82f7-f81eba9db204 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:05.798809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a7777-af83-4f3e-ac73-b58af6c957a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:05.799486 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a7777-af83-4f3e-ac73-b58af6c957a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:05.800448 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a7777-af83-4f3e-ac73-b58af6c957a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:05.800727 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a7777-af83-4f3e-ac73-b58af6c957a8 admin admin] PATH_INFO: `/v3/projects/ff4b17d24aa448d59c0a032845fb803b/users/5747f79cddf44bf4a0df8cfc585d6265/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:05.818056 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a7777-af83-4f3e-ac73-b58af6c957a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff4b17d24aa448d59c0a032845fb803b, user_id=5747f79cddf44bf4a0df8cfc585d6265, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:05.820793 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a7777-af83-4f3e-ac73-b58af6c957a8 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:05.821485 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff61e3-966b-4e9f-8c25-ea6f2f0b7df2 tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f31ce6ec71942618f9f51e45ba0f9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826d67a82a934deca82a6556c703a644', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:05.822400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff61e3-966b-4e9f-8c25-ea6f2f0b7df2 tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:05.823103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff61e3-966b-4e9f-8c25-ea6f2f0b7df2 tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:05.823607 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff61e3-966b-4e9f-8c25-ea6f2f0b7df2 tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:05.825034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff61e3-966b-4e9f-8c25-ea6f2f0b7df2 tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:31:05.825584 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff61e3-966b-4e9f-8c25-ea6f2f0b7df2 tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:05.827552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff61e3-966b-4e9f-8c25-ea6f2f0b7df2 tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:05.839444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 994/2016] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:31:05 2022] PUT /identity/v3/projects/ff4b17d24aa448d59c0a032845fb803b/users/5747f79cddf44bf4a0df8cfc585d6265/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:05.839444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286168a7-3fe8-4c72-8354-de34a4cb37e0 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:05.839444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286168a7-3fe8-4c72-8354-de34a4cb37e0 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:05.839444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286168a7-3fe8-4c72-8354-de34a4cb37e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:05.847425 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1023/2017] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:31:05.847425 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff61e3-966b-4e9f-8c25-ea6f2f0b7df2 tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:05.859809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff61e3-966b-4e9f-8c25-ea6f2f0b7df2 tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:05.889796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-286168a7-3fe8-4c72-8354-de34a4cb37e0 None None] MFA Rules not processed for user `576ffe2ec3734f6fa202ef4eca81adad`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:05.891601 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-286168a7-3fe8-4c72-8354-de34a4cb37e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:05.941634 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1dfe86-3350-495c-bec3-1f642740a234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:05.941900 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1dfe86-3350-495c-bec3-1f642740a234 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:05.942068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1dfe86-3350-495c-bec3-1f642740a234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:05.942506 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1dfe86-3350-495c-bec3-1f642740a234 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:05.943316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1dfe86-3350-495c-bec3-1f642740a234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:05.945248 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1dfe86-3350-495c-bec3-1f642740a234 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:05.960363 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1024/2018] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:31:05.960363 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1dfe86-3350-495c-bec3-1f642740a234 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:05.983794 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 995/2019] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:05.989392 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286168a7-3fe8-4c72-8354-de34a4cb37e0 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:06.002012 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286168a7-3fe8-4c72-8354-de34a4cb37e0 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:06.005118 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:06.007694 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16244576-32eb-4085-829f-d204e82e58cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:06.008429 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16244576-32eb-4085-829f-d204e82e58cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:06.008726 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16244576-32eb-4085-829f-d204e82e58cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:06.009021 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16244576-32eb-4085-829f-d204e82e58cb admin admin] PATH_INFO: `/v3/projects/ff4b17d24aa448d59c0a032845fb803b/users/5747f79cddf44bf4a0df8cfc585d6265/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:06.011158 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16244576-32eb-4085-829f-d204e82e58cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff4b17d24aa448d59c0a032845fb803b, user_id=5747f79cddf44bf4a0df8cfc585d6265, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:06.013891 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16244576-32eb-4085-829f-d204e82e58cb admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:06.021219 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:06.027171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 996/2020] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:31:05 2022] PUT /identity/v3/projects/ff4b17d24aa448d59c0a032845fb803b/users/5747f79cddf44bf4a0df8cfc585d6265/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:06.036243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16244576-32eb-4085-829f-d204e82e58cb admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:06.049679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1dfe86-3350-495c-bec3-1f642740a234 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:06.105638 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16244576-32eb-4085-829f-d204e82e58cb admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:06.109884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af14ea4f-1de9-46b6-bd0e-1570dadafe97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:06.110598 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14ea4f-1de9-46b6-bd0e-1570dadafe97 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:06.110790 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14ea4f-1de9-46b6-bd0e-1570dadafe97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:06.111070 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14ea4f-1de9-46b6-bd0e-1570dadafe97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:06.120057 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31f972-dbd5-4366-892b-7b3172ccaf16 tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f31ce6ec71942618f9f51e45ba0f9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826d67a82a934deca82a6556c703a644', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:06.120948 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31f972-dbd5-4366-892b-7b3172ccaf16 tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:06.122150 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31f972-dbd5-4366-892b-7b3172ccaf16 tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:06.122616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31f972-dbd5-4366-892b-7b3172ccaf16 tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:06.124630 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31f972-dbd5-4366-892b-7b3172ccaf16 tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:06.126582 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31f972-dbd5-4366-892b-7b3172ccaf16 tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:06.171848 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1025/2021] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:31:05 2022] POST /identity/v3/projects => generated 354 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:31:06.173271 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c30ac-5332-43aa-a2b1-14367b641a45 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:06.173730 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c30ac-5332-43aa-a2b1-14367b641a45 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:06.174010 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c30ac-5332-43aa-a2b1-14367b641a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:06.196045 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af14ea4f-1de9-46b6-bd0e-1570dadafe97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:06.196848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af14ea4f-1de9-46b6-bd0e-1570dadafe97 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:06.207808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-193c30ac-5332-43aa-a2b1-14367b641a45 None None] MFA Rules not processed for user `5747f79cddf44bf4a0df8cfc585d6265`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:06.209196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-193c30ac-5332-43aa-a2b1-14367b641a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:06.224019 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 997/2022] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:31:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:31:06.227348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af14ea4f-1de9-46b6-bd0e-1570dadafe97 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:06.249617 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af14ea4f-1de9-46b6-bd0e-1570dadafe97 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:06.381706 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82ce9b-c966-4423-b6f3-44220a15f79a tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f31ce6ec71942618f9f51e45ba0f9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826d67a82a934deca82a6556c703a644', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:06.383611 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82ce9b-c966-4423-b6f3-44220a15f79a tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:06.383949 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82ce9b-c966-4423-b6f3-44220a15f79a tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:06.384803 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82ce9b-c966-4423-b6f3-44220a15f79a tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:06.386581 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82ce9b-c966-4423-b6f3-44220a15f79a tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:31:06.388288 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82ce9b-c966-4423-b6f3-44220a15f79a tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:06.389939 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1026/2023] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:06.393827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82ce9b-c966-4423-b6f3-44220a15f79a tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:06.405213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c30ac-5332-43aa-a2b1-14367b641a45 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:06.428353 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 998/2024] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:31:06.489671 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:06.506030 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c30ac-5332-43aa-a2b1-14367b641a45 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:06.510142 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96a355-6fcc-458a-afce-7801d702eafd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:06.510856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96a355-6fcc-458a-afce-7801d702eafd service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:06.511149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96a355-6fcc-458a-afce-7801d702eafd service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:06.511471 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96a355-6fcc-458a-afce-7801d702eafd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:06.530624 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96a355-6fcc-458a-afce-7801d702eafd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:06.532558 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96a355-6fcc-458a-afce-7801d702eafd service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:06.552545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1027/2025] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:31:06 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:31:06.643322 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82ce9b-c966-4423-b6f3-44220a15f79a tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:06.655471 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82ce9b-c966-4423-b6f3-44220a15f79a tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:06.660364 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ece36-cb1b-475e-9311-b3316e351dcb tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f31ce6ec71942618f9f51e45ba0f9ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '826d67a82a934deca82a6556c703a644', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:06.661049 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ece36-cb1b-475e-9311-b3316e351dcb tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:06.661342 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ece36-cb1b-475e-9311-b3316e351dcb tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:06.662186 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ece36-cb1b-475e-9311-b3316e351dcb tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] PATH_INFO: `/v3/projects/a7efa503fac747a0981ad939e58ff0fc` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:06.663439 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ece36-cb1b-475e-9311-b3316e351dcb tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a7efa503fac747a0981ad939e58ff0fc)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:06.665798 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ece36-cb1b-475e-9311-b3316e351dcb tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:06.766123 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:06.775877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-683ece36-cb1b-475e-9311-b3316e351dcb tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7efa503fac747a0981ad939e58ff0fc', 'request_id': 'req-683ece36-cb1b-475e-9311-b3316e351dcb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:06.785239 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-683ece36-cb1b-475e-9311-b3316e351dcb tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7efa503fac747a0981ad939e58ff0fc', 'request_id': 'req-683ece36-cb1b-475e-9311-b3316e351dcb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:06.795159 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-683ece36-cb1b-475e-9311-b3316e351dcb tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7efa503fac747a0981ad939e58ff0fc', 'request_id': 'req-683ece36-cb1b-475e-9311-b3316e351dcb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:06.862193 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-683ece36-cb1b-475e-9311-b3316e351dcb tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] The token cache is being invalidate because project a7efa503fac747a0981ad939e58ff0fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:31:06.863548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-683ece36-cb1b-475e-9311-b3316e351dcb tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:06.866940 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-683ece36-cb1b-475e-9311-b3316e351dcb tempest-VolumeQuotaClassesTest-343118597 tempest-VolumeQuotaClassesTest-343118597-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:06.876795 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 999/2026] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:31:06 2022] DELETE /identity/v3/projects/a7efa503fac747a0981ad939e58ff0fc => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:07.255488 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96a355-6fcc-458a-afce-7801d702eafd service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:07.257029 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6a96a355-6fcc-458a-afce-7801d702eafd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:07.342432 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96a355-6fcc-458a-afce-7801d702eafd service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:07.408758 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae43e57-e5c9-4ad6-bb1e-9e51af6b06f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:07.410033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae43e57-e5c9-4ad6-bb1e-9e51af6b06f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:07.410319 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae43e57-e5c9-4ad6-bb1e-9e51af6b06f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:07.410597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae43e57-e5c9-4ad6-bb1e-9e51af6b06f9 admin admin] PATH_INFO: `/v3/users/4eef477e773c4af58e471775ffaf8011` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:07.412678 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae43e57-e5c9-4ad6-bb1e-9e51af6b06f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4eef477e773c4af58e471775ffaf8011)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:07.414731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae43e57-e5c9-4ad6-bb1e-9e51af6b06f9 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:07.498573 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bae43e57-e5c9-4ad6-bb1e-9e51af6b06f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4eef477e773c4af58e471775ffaf8011'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:07.529677 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bae43e57-e5c9-4ad6-bb1e-9e51af6b06f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4eef477e773c4af58e471775ffaf8011'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:07.530553 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bae43e57-e5c9-4ad6-bb1e-9e51af6b06f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4eef477e773c4af58e471775ffaf8011'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:07.532760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bae43e57-e5c9-4ad6-bb1e-9e51af6b06f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4eef477e773c4af58e471775ffaf8011'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:07.553006 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bae43e57-e5c9-4ad6-bb1e-9e51af6b06f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4eef477e773c4af58e471775ffaf8011'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:07.572638 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1028/2027] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:31:07 2022] DELETE /identity/v3/users/4eef477e773c4af58e471775ffaf8011 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:07.578032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eab1c0-953f-496f-acf1-0f09c92b2d2d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:07.578780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eab1c0-953f-496f-acf1-0f09c92b2d2d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:07.579324 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eab1c0-953f-496f-acf1-0f09c92b2d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:07.621757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-e5eab1c0-953f-496f-acf1-0f09c92b2d2d None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:07.623258 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e5eab1c0-953f-496f-acf1-0f09c92b2d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:07.658439 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae43e57-e5c9-4ad6-bb1e-9e51af6b06f9 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:07.663771 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bae43e57-e5c9-4ad6-bb1e-9e51af6b06f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:07.704013 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:07.720611 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1000/2028] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:07.743302 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eab1c0-953f-496f-acf1-0f09c92b2d2d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:07.746907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e5eab1c0-953f-496f-acf1-0f09c92b2d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:07.836343 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae43e57-e5c9-4ad6-bb1e-9e51af6b06f9 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:07.851454 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84dfbd-a1b7-4693-9b75-ea1fa52c8d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:07.853190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84dfbd-a1b7-4693-9b75-ea1fa52c8d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:07.853483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84dfbd-a1b7-4693-9b75-ea1fa52c8d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:07.853764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84dfbd-a1b7-4693-9b75-ea1fa52c8d28 admin admin] PATH_INFO: `/v3/users/7823df24b0d64b0d93a6ee0e684ed483` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:07.858023 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c84dfbd-a1b7-4693-9b75-ea1fa52c8d28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7823df24b0d64b0d93a6ee0e684ed483)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:07.859924 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c84dfbd-a1b7-4693-9b75-ea1fa52c8d28 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:07.942800 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eab1c0-953f-496f-acf1-0f09c92b2d2d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:07.950327 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59380c-ca86-4609-830f-bce638bcd142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:07.952076 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59380c-ca86-4609-830f-bce638bcd142 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:07.952577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59380c-ca86-4609-830f-bce638bcd142 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:07.953001 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59380c-ca86-4609-830f-bce638bcd142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:07.964685 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59380c-ca86-4609-830f-bce638bcd142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:07.966332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59380c-ca86-4609-830f-bce638bcd142 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:07.980731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1001/2029] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:31:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:31:07.982004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8c84dfbd-a1b7-4693-9b75-ea1fa52c8d28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7823df24b0d64b0d93a6ee0e684ed483'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:08.012483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8c84dfbd-a1b7-4693-9b75-ea1fa52c8d28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7823df24b0d64b0d93a6ee0e684ed483'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:08.013249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8c84dfbd-a1b7-4693-9b75-ea1fa52c8d28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7823df24b0d64b0d93a6ee0e684ed483'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:08.013877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8c84dfbd-a1b7-4693-9b75-ea1fa52c8d28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7823df24b0d64b0d93a6ee0e684ed483'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:08.033539 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8c84dfbd-a1b7-4693-9b75-ea1fa52c8d28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7823df24b0d64b0d93a6ee0e684ed483'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:08.056126 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1029/2030] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:31:07 2022] DELETE /identity/v3/users/7823df24b0d64b0d93a6ee0e684ed483 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:08.061181 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3632c-9363-4386-8503-0c04831edfba None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:08.061682 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3632c-9363-4386-8503-0c04831edfba None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:08.062133 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3632c-9363-4386-8503-0c04831edfba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:08.109734 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ecc3632c-9363-4386-8503-0c04831edfba None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:08.111801 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ecc3632c-9363-4386-8503-0c04831edfba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:08.286686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1002/2031] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:08.326528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84dfbd-a1b7-4693-9b75-ea1fa52c8d28 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:08.329075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8c84dfbd-a1b7-4693-9b75-ea1fa52c8d28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:08.450800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84dfbd-a1b7-4693-9b75-ea1fa52c8d28 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:08.473234 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740815e9-6690-46ac-a7ad-b5b7ca1f7111 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:08.474688 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740815e9-6690-46ac-a7ad-b5b7ca1f7111 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:08.475078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740815e9-6690-46ac-a7ad-b5b7ca1f7111 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:08.475692 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740815e9-6690-46ac-a7ad-b5b7ca1f7111 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:08.538262 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740815e9-6690-46ac-a7ad-b5b7ca1f7111 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:08.542357 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740815e9-6690-46ac-a7ad-b5b7ca1f7111 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:08.583953 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1030/2032] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:31:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:31:08.775019 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3632c-9363-4386-8503-0c04831edfba None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:08.776499 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ecc3632c-9363-4386-8503-0c04831edfba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:08.810451 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3632c-9363-4386-8503-0c04831edfba None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:08.820681 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ec2b1-25c9-44ed-b4f2-aa20670c553a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:08.821530 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ec2b1-25c9-44ed-b4f2-aa20670c553a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:08.821952 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ec2b1-25c9-44ed-b4f2-aa20670c553a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:08.822364 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ec2b1-25c9-44ed-b4f2-aa20670c553a admin admin] PATH_INFO: `/v3/projects/64256c19c24d4efc90fa0ee6651a2a6e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:08.829326 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ec2b1-25c9-44ed-b4f2-aa20670c553a admin admin] RBAC: Authorizing `identity:delete_project(project_id=64256c19c24d4efc90fa0ee6651a2a6e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:08.832527 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ec2b1-25c9-44ed-b4f2-aa20670c553a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:08.862946 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:08.968496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4b3ec2b1-25c9-44ed-b4f2-aa20670c553a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64256c19c24d4efc90fa0ee6651a2a6e', 'request_id': 'req-4b3ec2b1-25c9-44ed-b4f2-aa20670c553a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:08.978593 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4b3ec2b1-25c9-44ed-b4f2-aa20670c553a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64256c19c24d4efc90fa0ee6651a2a6e', 'request_id': 'req-4b3ec2b1-25c9-44ed-b4f2-aa20670c553a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:08.996377 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4b3ec2b1-25c9-44ed-b4f2-aa20670c553a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64256c19c24d4efc90fa0ee6651a2a6e', 'request_id': 'req-4b3ec2b1-25c9-44ed-b4f2-aa20670c553a'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:09.026185 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4b3ec2b1-25c9-44ed-b4f2-aa20670c553a admin admin] The token cache is being invalidate because project 64256c19c24d4efc90fa0ee6651a2a6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:31:09.026705 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4b3ec2b1-25c9-44ed-b4f2-aa20670c553a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:09.027646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4b3ec2b1-25c9-44ed-b4f2-aa20670c553a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:09.030382 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1003/2033] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:31:08 2022] DELETE /identity/v3/projects/64256c19c24d4efc90fa0ee6651a2a6e => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:09.035251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740815e9-6690-46ac-a7ad-b5b7ca1f7111 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:09.048852 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740815e9-6690-46ac-a7ad-b5b7ca1f7111 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:09.133927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ec2b1-25c9-44ed-b4f2-aa20670c553a admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:09.135634 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4b3ec2b1-25c9-44ed-b4f2-aa20670c553a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:09.145274 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:09.212149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70822d1-a8db-4b1f-be9b-fd589b76c44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:09.213003 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70822d1-a8db-4b1f-be9b-fd589b76c44c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:09.213408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70822d1-a8db-4b1f-be9b-fd589b76c44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:09.214033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70822d1-a8db-4b1f-be9b-fd589b76c44c admin admin] PATH_INFO: `/v3/users/4f31ce6ec71942618f9f51e45ba0f9ec` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:09.217741 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70822d1-a8db-4b1f-be9b-fd589b76c44c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f31ce6ec71942618f9f51e45ba0f9ec)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:09.222532 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70822d1-a8db-4b1f-be9b-fd589b76c44c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:09.233765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ec2b1-25c9-44ed-b4f2-aa20670c553a admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:09.243561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf9f1f-33b7-41ac-8221-9711b77ea6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:09.244229 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf9f1f-33b7-41ac-8221-9711b77ea6a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:09.244656 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf9f1f-33b7-41ac-8221-9711b77ea6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:09.244833 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf9f1f-33b7-41ac-8221-9711b77ea6a1 admin admin] PATH_INFO: `/v3/projects/d74662f7d3bd41e7ac835755d212e877` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:09.246068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf9f1f-33b7-41ac-8221-9711b77ea6a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d74662f7d3bd41e7ac835755d212e877)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:09.248012 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf9f1f-33b7-41ac-8221-9711b77ea6a1 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:09.331017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c70822d1-a8db-4b1f-be9b-fd589b76c44c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f31ce6ec71942618f9f51e45ba0f9ec'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:09.348212 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c70822d1-a8db-4b1f-be9b-fd589b76c44c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f31ce6ec71942618f9f51e45ba0f9ec'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:09.349379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c70822d1-a8db-4b1f-be9b-fd589b76c44c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f31ce6ec71942618f9f51e45ba0f9ec'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:09.350065 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c70822d1-a8db-4b1f-be9b-fd589b76c44c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f31ce6ec71942618f9f51e45ba0f9ec'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:09.351907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-77bf9f1f-33b7-41ac-8221-9711b77ea6a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd74662f7d3bd41e7ac835755d212e877', 'request_id': 'req-77bf9f1f-33b7-41ac-8221-9711b77ea6a1'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:09.359010 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-77bf9f1f-33b7-41ac-8221-9711b77ea6a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd74662f7d3bd41e7ac835755d212e877', 'request_id': 'req-77bf9f1f-33b7-41ac-8221-9711b77ea6a1'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:09.360648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c70822d1-a8db-4b1f-be9b-fd589b76c44c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f31ce6ec71942618f9f51e45ba0f9ec'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:09.375622 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-77bf9f1f-33b7-41ac-8221-9711b77ea6a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd74662f7d3bd41e7ac835755d212e877', 'request_id': 'req-77bf9f1f-33b7-41ac-8221-9711b77ea6a1'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:09.384191 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1031/2034] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:31:09 2022] DELETE /identity/v3/users/4f31ce6ec71942618f9f51e45ba0f9ec => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:09.398009 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-77bf9f1f-33b7-41ac-8221-9711b77ea6a1 admin admin] The token cache is being invalidate because project d74662f7d3bd41e7ac835755d212e877 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:31:09.398426 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-77bf9f1f-33b7-41ac-8221-9711b77ea6a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:09.399796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-77bf9f1f-33b7-41ac-8221-9711b77ea6a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:09.402004 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1004/2035] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:31:09 2022] DELETE /identity/v3/projects/d74662f7d3bd41e7ac835755d212e877 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:09.406124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70822d1-a8db-4b1f-be9b-fd589b76c44c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:09.407548 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c70822d1-a8db-4b1f-be9b-fd589b76c44c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:09.484495 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70822d1-a8db-4b1f-be9b-fd589b76c44c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:09.491226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f19fd-8098-41be-b3b8-f020b77a3180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:09.491884 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f19fd-8098-41be-b3b8-f020b77a3180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:09.492163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f19fd-8098-41be-b3b8-f020b77a3180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:09.492480 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f19fd-8098-41be-b3b8-f020b77a3180 admin admin] PATH_INFO: `/v3/users/5747f79cddf44bf4a0df8cfc585d6265` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:09.493972 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f19fd-8098-41be-b3b8-f020b77a3180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5747f79cddf44bf4a0df8cfc585d6265)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:09.495963 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f19fd-8098-41be-b3b8-f020b77a3180 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:09.561946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3f0f19fd-8098-41be-b3b8-f020b77a3180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5747f79cddf44bf4a0df8cfc585d6265'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:09.570339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:09.576672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3f0f19fd-8098-41be-b3b8-f020b77a3180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5747f79cddf44bf4a0df8cfc585d6265'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:09.577687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3f0f19fd-8098-41be-b3b8-f020b77a3180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5747f79cddf44bf4a0df8cfc585d6265'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:09.578846 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3f0f19fd-8098-41be-b3b8-f020b77a3180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5747f79cddf44bf4a0df8cfc585d6265'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:09.590875 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3f0f19fd-8098-41be-b3b8-f020b77a3180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5747f79cddf44bf4a0df8cfc585d6265'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:09.613156 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1032/2036] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:31:09 2022] DELETE /identity/v3/users/5747f79cddf44bf4a0df8cfc585d6265 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:31:09.738466 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf9f1f-33b7-41ac-8221-9711b77ea6a1 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:09.740166 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-77bf9f1f-33b7-41ac-8221-9711b77ea6a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:09.784246 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:09.833759 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf9f1f-33b7-41ac-8221-9711b77ea6a1 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:09.914325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24d626-470b-4fe0-b101-2206b95ad1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:09.915556 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24d626-470b-4fe0-b101-2206b95ad1a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:09.915915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24d626-470b-4fe0-b101-2206b95ad1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:09.916266 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24d626-470b-4fe0-b101-2206b95ad1a2 admin admin] PATH_INFO: `/v3/projects/826d67a82a934deca82a6556c703a644` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:09.918297 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24d626-470b-4fe0-b101-2206b95ad1a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=826d67a82a934deca82a6556c703a644)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:09.920995 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24d626-470b-4fe0-b101-2206b95ad1a2 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:09.960014 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f19fd-8098-41be-b3b8-f020b77a3180 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:09.962016 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3f0f19fd-8098-41be-b3b8-f020b77a3180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:09.997968 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f19fd-8098-41be-b3b8-f020b77a3180 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:10.030946 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ab24d626-470b-4fe0-b101-2206b95ad1a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '826d67a82a934deca82a6556c703a644', 'request_id': 'req-ab24d626-470b-4fe0-b101-2206b95ad1a2'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:10.041242 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ab24d626-470b-4fe0-b101-2206b95ad1a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '826d67a82a934deca82a6556c703a644', 'request_id': 'req-ab24d626-470b-4fe0-b101-2206b95ad1a2'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:10.057505 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ab24d626-470b-4fe0-b101-2206b95ad1a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '826d67a82a934deca82a6556c703a644', 'request_id': 'req-ab24d626-470b-4fe0-b101-2206b95ad1a2'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:10.072176 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ab24d626-470b-4fe0-b101-2206b95ad1a2 admin admin] The token cache is being invalidate because project 826d67a82a934deca82a6556c703a644 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:31:10.072569 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ab24d626-470b-4fe0-b101-2206b95ad1a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:10.073226 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ab24d626-470b-4fe0-b101-2206b95ad1a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:10.075098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1005/2037] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:31:09 2022] DELETE /identity/v3/projects/826d67a82a934deca82a6556c703a644 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:10.080158 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec893e6-83d0-41c5-a73b-8419047cd3b6 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:10.081867 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec893e6-83d0-41c5-a73b-8419047cd3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:10.082239 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec893e6-83d0-41c5-a73b-8419047cd3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:10.094011 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795bea0c-f5ba-4483-a222-8e38d69a8a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:10.095117 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bea0c-f5ba-4483-a222-8e38d69a8a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:10.095472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bea0c-f5ba-4483-a222-8e38d69a8a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:10.095750 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bea0c-f5ba-4483-a222-8e38d69a8a17 admin admin] PATH_INFO: `/v3/projects/ff4b17d24aa448d59c0a032845fb803b` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:10.097814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795bea0c-f5ba-4483-a222-8e38d69a8a17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff4b17d24aa448d59c0a032845fb803b)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:10.099750 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795bea0c-f5ba-4483-a222-8e38d69a8a17 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:10.107560 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-dec893e6-83d0-41c5-a73b-8419047cd3b6 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:10.110048 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-dec893e6-83d0-41c5-a73b-8419047cd3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:10.131787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1006/2038] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:10.131787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a3cc9-355c-4e89-965a-0b9e4ebd4bde None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:10.131787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a3cc9-355c-4e89-965a-0b9e4ebd4bde None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:10.131787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a3cc9-355c-4e89-965a-0b9e4ebd4bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:10.152700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-f17a3cc9-355c-4e89-965a-0b9e4ebd4bde None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:10.154001 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f17a3cc9-355c-4e89-965a-0b9e4ebd4bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:10.162606 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-795bea0c-f5ba-4483-a222-8e38d69a8a17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff4b17d24aa448d59c0a032845fb803b', 'request_id': 'req-795bea0c-f5ba-4483-a222-8e38d69a8a17'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:10.170324 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-795bea0c-f5ba-4483-a222-8e38d69a8a17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff4b17d24aa448d59c0a032845fb803b', 'request_id': 'req-795bea0c-f5ba-4483-a222-8e38d69a8a17'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:10.180790 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-795bea0c-f5ba-4483-a222-8e38d69a8a17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff4b17d24aa448d59c0a032845fb803b', 'request_id': 'req-795bea0c-f5ba-4483-a222-8e38d69a8a17'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:10.198825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-795bea0c-f5ba-4483-a222-8e38d69a8a17 admin admin] The token cache is being invalidate because project ff4b17d24aa448d59c0a032845fb803b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:31:10.199358 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-795bea0c-f5ba-4483-a222-8e38d69a8a17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:10.200513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-795bea0c-f5ba-4483-a222-8e38d69a8a17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:10.202740 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1033/2039] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:31:09 2022] DELETE /identity/v3/projects/ff4b17d24aa448d59c0a032845fb803b => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:10.207728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235e8a6-2387-42bc-ae41-009faf4ea58b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:10.208204 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235e8a6-2387-42bc-ae41-009faf4ea58b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:10.208656 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235e8a6-2387-42bc-ae41-009faf4ea58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:10.220612 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:10.237729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d235e8a6-2387-42bc-ae41-009faf4ea58b None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:10.241927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d235e8a6-2387-42bc-ae41-009faf4ea58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:10.256789 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1034/2040] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:10.259082 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1007/2041] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:10.260518 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b0248-e2bf-4d4b-a86d-e8a35499df82 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:10.261060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b0248-e2bf-4d4b-a86d-e8a35499df82 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:10.261483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b0248-e2bf-4d4b-a86d-e8a35499df82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:10.263162 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a3cc9-355c-4e89-965a-0b9e4ebd4bde None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:10.273574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a3cc9-355c-4e89-965a-0b9e4ebd4bde None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:10.285032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21007c7f-3dcf-42fd-90f9-9eac8e157fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:10.286131 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21007c7f-3dcf-42fd-90f9-9eac8e157fff admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:10.286256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f92b0248-e2bf-4d4b-a86d-e8a35499df82 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:10.287082 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21007c7f-3dcf-42fd-90f9-9eac8e157fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:10.287583 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21007c7f-3dcf-42fd-90f9-9eac8e157fff admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:10.287690 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f92b0248-e2bf-4d4b-a86d-e8a35499df82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:10.288643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21007c7f-3dcf-42fd-90f9-9eac8e157fff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:31:10.289136 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21007c7f-3dcf-42fd-90f9-9eac8e157fff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:10.291344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21007c7f-3dcf-42fd-90f9-9eac8e157fff admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:10.300347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1035/2042] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:31:10.304044 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b0248-e2bf-4d4b-a86d-e8a35499df82 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:10.315176 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1008/2043] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:31:10.324289 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21007c7f-3dcf-42fd-90f9-9eac8e157fff admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:10.330077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b0248-e2bf-4d4b-a86d-e8a35499df82 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:10.334511 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21007c7f-3dcf-42fd-90f9-9eac8e157fff admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:10.337648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4a0f1-a3be-4b65-911f-701c3874c1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:10.338473 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4a0f1-a3be-4b65-911f-701c3874c1c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:10.338873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4a0f1-a3be-4b65-911f-701c3874c1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:10.339207 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4a0f1-a3be-4b65-911f-701c3874c1c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:10.340701 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4a0f1-a3be-4b65-911f-701c3874c1c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:31:10.340701 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4a0f1-a3be-4b65-911f-701c3874c1c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:10.342457 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4a0f1-a3be-4b65-911f-701c3874c1c0 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:10.349391 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71645d-4e9e-4aa7-b4e8-6eb80784f950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:10.351562 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71645d-4e9e-4aa7-b4e8-6eb80784f950 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:10.352037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71645d-4e9e-4aa7-b4e8-6eb80784f950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:10.353237 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71645d-4e9e-4aa7-b4e8-6eb80784f950 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:10.354226 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71645d-4e9e-4aa7-b4e8-6eb80784f950 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:10.356848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71645d-4e9e-4aa7-b4e8-6eb80784f950 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:10.361598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1036/2044] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:31:10.362636 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4a0f1-a3be-4b65-911f-701c3874c1c0 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:10.376730 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4a0f1-a3be-4b65-911f-701c3874c1c0 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:10.389467 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ab308-bb31-4132-95cf-23385982cbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:10.390164 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ab308-bb31-4132-95cf-23385982cbf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:10.390567 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ab308-bb31-4132-95cf-23385982cbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:10.391124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ab308-bb31-4132-95cf-23385982cbf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:10.391893 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ab308-bb31-4132-95cf-23385982cbf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:10.394828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ab308-bb31-4132-95cf-23385982cbf7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:10.403499 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1009/2045] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:31:10 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:31:10.407464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71645d-4e9e-4aa7-b4e8-6eb80784f950 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:10.442462 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1037/2046] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:31:10 2022] POST /identity/v3/projects => generated 373 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:31:10.443465 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ab308-bb31-4132-95cf-23385982cbf7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:10.470154 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:10.521375 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71645d-4e9e-4aa7-b4e8-6eb80784f950 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:10.538901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ab308-bb31-4132-95cf-23385982cbf7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:10.545236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff564098-5a69-4816-b1c2-d245968f16dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:10.545852 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff564098-5a69-4816-b1c2-d245968f16dc admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:10.546133 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff564098-5a69-4816-b1c2-d245968f16dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:10.546408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff564098-5a69-4816-b1c2-d245968f16dc admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:10.547474 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff564098-5a69-4816-b1c2-d245968f16dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:10.550442 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff564098-5a69-4816-b1c2-d245968f16dc admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:10.573374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1038/2047] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:31:10 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:31:10.577795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff564098-5a69-4816-b1c2-d245968f16dc admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:10.589843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff564098-5a69-4816-b1c2-d245968f16dc admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:10.594681 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527245ce-e9ea-41e8-8d74-fc1fe9c749e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:10.594842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b798476-5402-498a-a1fa-c37d59b47443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:10.595300 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527245ce-e9ea-41e8-8d74-fc1fe9c749e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:10.595527 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b798476-5402-498a-a1fa-c37d59b47443 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:10.595639 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527245ce-e9ea-41e8-8d74-fc1fe9c749e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:10.595842 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527245ce-e9ea-41e8-8d74-fc1fe9c749e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:10.595950 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b798476-5402-498a-a1fa-c37d59b47443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:10.596238 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b798476-5402-498a-a1fa-c37d59b47443 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:10.596609 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527245ce-e9ea-41e8-8d74-fc1fe9c749e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:10.597115 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b798476-5402-498a-a1fa-c37d59b47443 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:10.598211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527245ce-e9ea-41e8-8d74-fc1fe9c749e1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:10.599292 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b798476-5402-498a-a1fa-c37d59b47443 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:10.609921 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1039/2048] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:31:10.614075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527245ce-e9ea-41e8-8d74-fc1fe9c749e1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:10.626318 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527245ce-e9ea-41e8-8d74-fc1fe9c749e1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:10.631366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1010/2049] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:31:10 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 12:31:10.631496 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907c103-37f2-4572-856a-e300ce891fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:10.631940 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907c103-37f2-4572-856a-e300ce891fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:10.632226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907c103-37f2-4572-856a-e300ce891fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:10.632538 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907c103-37f2-4572-856a-e300ce891fc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:10.632724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b798476-5402-498a-a1fa-c37d59b47443 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:10.633234 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907c103-37f2-4572-856a-e300ce891fc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:10.634805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907c103-37f2-4572-856a-e300ce891fc5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:10.642612 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b798476-5402-498a-a1fa-c37d59b47443 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:10.646426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1040/2050] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:31:10.647818 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7926f1-bc88-4a0e-a919-03dbbbee895a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:10.648465 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7926f1-bc88-4a0e-a919-03dbbbee895a admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:10.648747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7926f1-bc88-4a0e-a919-03dbbbee895a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:10.649019 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7926f1-bc88-4a0e-a919-03dbbbee895a admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:10.650003 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7926f1-bc88-4a0e-a919-03dbbbee895a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:10.650403 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907c103-37f2-4572-856a-e300ce891fc5 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:10.651641 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7926f1-bc88-4a0e-a919-03dbbbee895a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:10.660572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907c103-37f2-4572-856a-e300ce891fc5 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:10.662661 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1011/2051] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:31:10.665735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac86140-ddad-4b25-b777-848ee6bab335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:10.666211 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7926f1-bc88-4a0e-a919-03dbbbee895a admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:10.666394 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac86140-ddad-4b25-b777-848ee6bab335 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:10.666679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac86140-ddad-4b25-b777-848ee6bab335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:10.666951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac86140-ddad-4b25-b777-848ee6bab335 admin admin] PATH_INFO: `/v3/projects/9d772894bb6a4e52a787eff5176016db/users/6619f2854db44a1f844c492b193a9ed5/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:10.678650 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7926f1-bc88-4a0e-a919-03dbbbee895a admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:10.682314 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac86140-ddad-4b25-b777-848ee6bab335 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d772894bb6a4e52a787eff5176016db, user_id=6619f2854db44a1f844c492b193a9ed5, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:10.685122 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79cb06a-e2f5-42d1-89a7-7cddd2a71f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:10.685310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac86140-ddad-4b25-b777-848ee6bab335 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:10.685747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cb06a-e2f5-42d1-89a7-7cddd2a71f31 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:10.686028 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cb06a-e2f5-42d1-89a7-7cddd2a71f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:10.686298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cb06a-e2f5-42d1-89a7-7cddd2a71f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:10.686991 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79cb06a-e2f5-42d1-89a7-7cddd2a71f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:10.688894 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79cb06a-e2f5-42d1-89a7-7cddd2a71f31 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:10.700581 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1012/2052] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 12:31:10.701208 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1041/2053] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:31:10 2022] PUT /identity/v3/projects/9d772894bb6a4e52a787eff5176016db/users/6619f2854db44a1f844c492b193a9ed5/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:31:10.711119 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79cb06a-e2f5-42d1-89a7-7cddd2a71f31 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:10.712612 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cec126-2bd4-4360-bb11-015a9765179e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:10.712914 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cec126-2bd4-4360-bb11-015a9765179e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:10.713192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cec126-2bd4-4360-bb11-015a9765179e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:10.717144 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79cb06a-e2f5-42d1-89a7-7cddd2a71f31 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:10.744266 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-96cec126-2bd4-4360-bb11-015a9765179e None None] MFA Rules not processed for user `6619f2854db44a1f844c492b193a9ed5`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:10.745882 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-96cec126-2bd4-4360-bb11-015a9765179e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:10.752642 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:10.777585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f96cca-35f6-4a32-acfc-8e04b7d4654e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:10.778251 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f96cca-35f6-4a32-acfc-8e04b7d4654e admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:10.778555 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f96cca-35f6-4a32-acfc-8e04b7d4654e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:10.778830 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f96cca-35f6-4a32-acfc-8e04b7d4654e admin admin] PATH_INFO: `/v3/projects/8858f4383248499182026a12000cadb4/users/2d60a1a81aaa4b9cb23b6c0305197d95/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:10.793437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f96cca-35f6-4a32-acfc-8e04b7d4654e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8858f4383248499182026a12000cadb4, user_id=2d60a1a81aaa4b9cb23b6c0305197d95, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:10.796452 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f96cca-35f6-4a32-acfc-8e04b7d4654e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:10.809398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1013/2054] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:31:10 2022] PUT /identity/v3/projects/8858f4383248499182026a12000cadb4/users/2d60a1a81aaa4b9cb23b6c0305197d95/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:31:10.810720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1042/2055] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:10.821395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e277ebf-d10a-4d43-b916-2050ccb2f8e5 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:10.821836 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e277ebf-d10a-4d43-b916-2050ccb2f8e5 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:10.822114 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e277ebf-d10a-4d43-b916-2050ccb2f8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:10.822713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cec126-2bd4-4360-bb11-015a9765179e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:10.824116 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-96cec126-2bd4-4360-bb11-015a9765179e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:10.850422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-9e277ebf-d10a-4d43-b916-2050ccb2f8e5 None None] MFA Rules not processed for user `2d60a1a81aaa4b9cb23b6c0305197d95`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:10.851625 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9e277ebf-d10a-4d43-b916-2050ccb2f8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:10.879403 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:10.883768 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cec126-2bd4-4360-bb11-015a9765179e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:10.920036 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1014/2056] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:10.931873 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e277ebf-d10a-4d43-b916-2050ccb2f8e5 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:10.933229 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9e277ebf-d10a-4d43-b916-2050ccb2f8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:10.933890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509b434-53a9-4c16-a06b-61a25037e229 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:10.934508 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509b434-53a9-4c16-a06b-61a25037e229 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:10.934784 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509b434-53a9-4c16-a06b-61a25037e229 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:10.935053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509b434-53a9-4c16-a06b-61a25037e229 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:10.945208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e277ebf-d10a-4d43-b916-2050ccb2f8e5 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:10.949000 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934d956-2a18-4d3d-90d9-fc14c6db8f94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:10.949611 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934d956-2a18-4d3d-90d9-fc14c6db8f94 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:10.949892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934d956-2a18-4d3d-90d9-fc14c6db8f94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:10.950280 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934d956-2a18-4d3d-90d9-fc14c6db8f94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:10.958661 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934d956-2a18-4d3d-90d9-fc14c6db8f94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:10.960083 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934d956-2a18-4d3d-90d9-fc14c6db8f94 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:10.968818 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1015/2057] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:31:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:31:10.989856 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934d956-2a18-4d3d-90d9-fc14c6db8f94 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:10.997724 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0509b434-53a9-4c16-a06b-61a25037e229 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:10.999182 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0509b434-53a9-4c16-a06b-61a25037e229 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:11.008891 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1043/2058] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:31:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:31:11.036476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509b434-53a9-4c16-a06b-61a25037e229 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:11.068461 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934d956-2a18-4d3d-90d9-fc14c6db8f94 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:11.073190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0ed3a-3ca2-41c9-84fd-49f3371effe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:11.073839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0ed3a-3ca2-41c9-84fd-49f3371effe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:11.074121 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0ed3a-3ca2-41c9-84fd-49f3371effe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:11.074399 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0ed3a-3ca2-41c9-84fd-49f3371effe5 admin admin] PATH_INFO: `/v3/users/2d60a1a81aaa4b9cb23b6c0305197d95` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:11.075578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0ed3a-3ca2-41c9-84fd-49f3371effe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d60a1a81aaa4b9cb23b6c0305197d95)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:11.077530 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0ed3a-3ca2-41c9-84fd-49f3371effe5 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:11.108965 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509b434-53a9-4c16-a06b-61a25037e229 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:11.114824 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b540c-1ea4-4dd9-acac-3f4883e81f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:11.115802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b540c-1ea4-4dd9-acac-3f4883e81f1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:11.116149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b540c-1ea4-4dd9-acac-3f4883e81f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:11.116461 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b540c-1ea4-4dd9-acac-3f4883e81f1f admin admin] PATH_INFO: `/v3/users/6619f2854db44a1f844c492b193a9ed5` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:11.117695 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b540c-1ea4-4dd9-acac-3f4883e81f1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6619f2854db44a1f844c492b193a9ed5)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:11.119699 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b540c-1ea4-4dd9-acac-3f4883e81f1f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:11.122499 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:11.131642 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-faf0ed3a-3ca2-41c9-84fd-49f3371effe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d60a1a81aaa4b9cb23b6c0305197d95'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:11.144545 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-faf0ed3a-3ca2-41c9-84fd-49f3371effe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d60a1a81aaa4b9cb23b6c0305197d95'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:11.159288 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-faf0ed3a-3ca2-41c9-84fd-49f3371effe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d60a1a81aaa4b9cb23b6c0305197d95'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:11.160142 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-faf0ed3a-3ca2-41c9-84fd-49f3371effe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d60a1a81aaa4b9cb23b6c0305197d95'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:11.160941 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-faf0ed3a-3ca2-41c9-84fd-49f3371effe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d60a1a81aaa4b9cb23b6c0305197d95'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:11.176992 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1016/2059] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:31:10 2022] DELETE /identity/v3/users/2d60a1a81aaa4b9cb23b6c0305197d95 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:31:11.179208 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8e6b540c-1ea4-4dd9-acac-3f4883e81f1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6619f2854db44a1f844c492b193a9ed5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:11.181259 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc29313-bea5-4777-8786-cd9511272ff0 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:11.181549 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc29313-bea5-4777-8786-cd9511272ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:11.181827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc29313-bea5-4777-8786-cd9511272ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:11.194665 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:11.200547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8e6b540c-1ea4-4dd9-acac-3f4883e81f1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6619f2854db44a1f844c492b193a9ed5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:11.201323 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8e6b540c-1ea4-4dd9-acac-3f4883e81f1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6619f2854db44a1f844c492b193a9ed5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:11.201951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8e6b540c-1ea4-4dd9-acac-3f4883e81f1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6619f2854db44a1f844c492b193a9ed5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:11.202946 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ffc29313-bea5-4777-8786-cd9511272ff0 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:11.204133 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ffc29313-bea5-4777-8786-cd9511272ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:11.209715 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8e6b540c-1ea4-4dd9-acac-3f4883e81f1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6619f2854db44a1f844c492b193a9ed5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:11.222582 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1044/2060] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:31:11 2022] DELETE /identity/v3/users/6619f2854db44a1f844c492b193a9ed5 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:11.226817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45e227-af95-4bf4-a34f-3f0b38d65b9e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:11.227221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45e227-af95-4bf4-a34f-3f0b38d65b9e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:11.227501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45e227-af95-4bf4-a34f-3f0b38d65b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:11.247372 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-7b45e227-af95-4bf4-a34f-3f0b38d65b9e None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:11.248598 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7b45e227-af95-4bf4-a34f-3f0b38d65b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:11.264005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1017/2061] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:31:11.274491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc29313-bea5-4777-8786-cd9511272ff0 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:11.275904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ffc29313-bea5-4777-8786-cd9511272ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:11.311227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1045/2062] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:11.321999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45e227-af95-4bf4-a34f-3f0b38d65b9e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:11.323391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7b45e227-af95-4bf4-a34f-3f0b38d65b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:11.332353 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc29313-bea5-4777-8786-cd9511272ff0 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:11.381173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8abf5-aaa7-4568-848a-5ee3038644dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:11.381791 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8abf5-aaa7-4568-848a-5ee3038644dc service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:11.382069 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8abf5-aaa7-4568-848a-5ee3038644dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:11.382341 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8abf5-aaa7-4568-848a-5ee3038644dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:11.386267 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45e227-af95-4bf4-a34f-3f0b38d65b9e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:11.389912 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a36f3-63ae-4a34-a3b7-e7cff1ba8793 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:11.390521 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a36f3-63ae-4a34-a3b7-e7cff1ba8793 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:11.390800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a36f3-63ae-4a34-a3b7-e7cff1ba8793 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:11.391071 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a36f3-63ae-4a34-a3b7-e7cff1ba8793 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:11.391617 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8abf5-aaa7-4568-848a-5ee3038644dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:11.393067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8abf5-aaa7-4568-848a-5ee3038644dc service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:11.400505 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a36f3-63ae-4a34-a3b7-e7cff1ba8793 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:11.401951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a36f3-63ae-4a34-a3b7-e7cff1ba8793 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:11.403422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1018/2063] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:31:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:31:11.412840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1046/2064] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:31:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:31:11.889876 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8abf5-aaa7-4568-848a-5ee3038644dc service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:11.891306 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-eea8abf5-aaa7-4568-848a-5ee3038644dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:11.898392 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a36f3-63ae-4a34-a3b7-e7cff1ba8793 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:11.900018 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-957a36f3-63ae-4a34-a3b7-e7cff1ba8793 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:11.907667 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8abf5-aaa7-4568-848a-5ee3038644dc service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:11.913655 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd9825-90c3-4bed-a007-8b2b9dfe3c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:11.914362 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd9825-90c3-4bed-a007-8b2b9dfe3c7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:11.914713 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd9825-90c3-4bed-a007-8b2b9dfe3c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:11.914910 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd9825-90c3-4bed-a007-8b2b9dfe3c7d admin admin] PATH_INFO: `/v3/projects/8858f4383248499182026a12000cadb4` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:11.916058 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd9825-90c3-4bed-a007-8b2b9dfe3c7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8858f4383248499182026a12000cadb4)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:11.918076 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd9825-90c3-4bed-a007-8b2b9dfe3c7d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:11.923119 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a36f3-63ae-4a34-a3b7-e7cff1ba8793 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:11.931204 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:11.932332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e8585-faa3-4e3b-8141-f04683463ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:11.933129 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e8585-faa3-4e3b-8141-f04683463ca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:11.933129 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e8585-faa3-4e3b-8141-f04683463ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:11.933593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e8585-faa3-4e3b-8141-f04683463ca4 admin admin] PATH_INFO: `/v3/projects/9d772894bb6a4e52a787eff5176016db` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:11.935210 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e8585-faa3-4e3b-8141-f04683463ca4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d772894bb6a4e52a787eff5176016db)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:11.937689 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e8585-faa3-4e3b-8141-f04683463ca4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:11.976992 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-15dd9825-90c3-4bed-a007-8b2b9dfe3c7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8858f4383248499182026a12000cadb4', 'request_id': 'req-15dd9825-90c3-4bed-a007-8b2b9dfe3c7d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:11.982967 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-15dd9825-90c3-4bed-a007-8b2b9dfe3c7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8858f4383248499182026a12000cadb4', 'request_id': 'req-15dd9825-90c3-4bed-a007-8b2b9dfe3c7d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:11.990645 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-924e8585-faa3-4e3b-8141-f04683463ca4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d772894bb6a4e52a787eff5176016db', 'request_id': 'req-924e8585-faa3-4e3b-8141-f04683463ca4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:11.998142 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-924e8585-faa3-4e3b-8141-f04683463ca4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d772894bb6a4e52a787eff5176016db', 'request_id': 'req-924e8585-faa3-4e3b-8141-f04683463ca4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:12.003089 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-15dd9825-90c3-4bed-a007-8b2b9dfe3c7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8858f4383248499182026a12000cadb4', 'request_id': 'req-15dd9825-90c3-4bed-a007-8b2b9dfe3c7d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:12.009297 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-924e8585-faa3-4e3b-8141-f04683463ca4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d772894bb6a4e52a787eff5176016db', 'request_id': 'req-924e8585-faa3-4e3b-8141-f04683463ca4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:12.015658 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-15dd9825-90c3-4bed-a007-8b2b9dfe3c7d admin admin] The token cache is being invalidate because project 8858f4383248499182026a12000cadb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:31:12.015950 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-15dd9825-90c3-4bed-a007-8b2b9dfe3c7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:12.016623 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-15dd9825-90c3-4bed-a007-8b2b9dfe3c7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:12.018889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1019/2065] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:31:11 2022] DELETE /identity/v3/projects/8858f4383248499182026a12000cadb4 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:12.023572 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372eee8-f14d-4403-aa4c-3a98dd305a6a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:12.023859 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372eee8-f14d-4403-aa4c-3a98dd305a6a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:12.024139 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372eee8-f14d-4403-aa4c-3a98dd305a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:12.024702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-924e8585-faa3-4e3b-8141-f04683463ca4 admin admin] The token cache is being invalidate because project 9d772894bb6a4e52a787eff5176016db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:31:12.025017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-924e8585-faa3-4e3b-8141-f04683463ca4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:12.025681 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-924e8585-faa3-4e3b-8141-f04683463ca4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:12.027195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1047/2066] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:31:11 2022] DELETE /identity/v3/projects/9d772894bb6a4e52a787eff5176016db => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:31:12.031821 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0edd58-7d60-486a-89b2-940eaa5e56bc None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:12.032041 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0edd58-7d60-486a-89b2-940eaa5e56bc None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:12.032344 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0edd58-7d60-486a-89b2-940eaa5e56bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:12.041359 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:12.044495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-2372eee8-f14d-4403-aa4c-3a98dd305a6a None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:12.045757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2372eee8-f14d-4403-aa4c-3a98dd305a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:12.055633 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0d0edd58-7d60-486a-89b2-940eaa5e56bc None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:12.056858 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0d0edd58-7d60-486a-89b2-940eaa5e56bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:12.113405 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1020/2067] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:12.121735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9166ff-d2da-4477-b140-ed466e08f36c None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:12.122024 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9166ff-d2da-4477-b140-ed466e08f36c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:12.122301 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9166ff-d2da-4477-b140-ed466e08f36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:12.123688 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1048/2068] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:31:12.131826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd62333-9739-4ad6-ac9e-de78d1fb6710 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:12.132177 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd62333-9739-4ad6-ac9e-de78d1fb6710 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:12.132480 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd62333-9739-4ad6-ac9e-de78d1fb6710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:12.142758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-0b9166ff-d2da-4477-b140-ed466e08f36c None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:12.144507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0b9166ff-d2da-4477-b140-ed466e08f36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:12.151889 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6cd62333-9739-4ad6-ac9e-de78d1fb6710 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:12.152548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1021/2069] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:31:12.155810 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6cd62333-9739-4ad6-ac9e-de78d1fb6710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:12.156663 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9166ff-d2da-4477-b140-ed466e08f36c None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:12.162148 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1049/2070] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:12.165982 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd62333-9739-4ad6-ac9e-de78d1fb6710 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:12.166153 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9166ff-d2da-4477-b140-ed466e08f36c None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:12.170637 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed66b7-3ad7-4fb6-8ca9-0277de3e2b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:12.171227 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed66b7-3ad7-4fb6-8ca9-0277de3e2b10 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:12.171502 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed66b7-3ad7-4fb6-8ca9-0277de3e2b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:12.171769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed66b7-3ad7-4fb6-8ca9-0277de3e2b10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:12.172477 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed66b7-3ad7-4fb6-8ca9-0277de3e2b10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:31:12.172781 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed66b7-3ad7-4fb6-8ca9-0277de3e2b10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:12.174365 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed66b7-3ad7-4fb6-8ca9-0277de3e2b10 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:12.174904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd62333-9739-4ad6-ac9e-de78d1fb6710 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:12.179478 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9548b7-16f0-41ca-8eaa-035b236bb31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:12.180067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9548b7-16f0-41ca-8eaa-035b236bb31d admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:12.180370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9548b7-16f0-41ca-8eaa-035b236bb31d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:12.180643 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9548b7-16f0-41ca-8eaa-035b236bb31d admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:12.181323 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9548b7-16f0-41ca-8eaa-035b236bb31d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:31:12.181619 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9548b7-16f0-41ca-8eaa-035b236bb31d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:12.183196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9548b7-16f0-41ca-8eaa-035b236bb31d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:12.189351 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1022/2071] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:31:12.190814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed66b7-3ad7-4fb6-8ca9-0277de3e2b10 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:12.201912 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed66b7-3ad7-4fb6-8ca9-0277de3e2b10 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:12.208690 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97193e5-76b4-44a4-90ff-998078be98a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:12.209508 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97193e5-76b4-44a4-90ff-998078be98a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:12.209940 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97193e5-76b4-44a4-90ff-998078be98a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:12.210158 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97193e5-76b4-44a4-90ff-998078be98a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:12.210432 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1050/2072] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:31:12.211016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97193e5-76b4-44a4-90ff-998078be98a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:12.213060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97193e5-76b4-44a4-90ff-998078be98a3 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:12.214784 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9548b7-16f0-41ca-8eaa-035b236bb31d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:12.227520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9548b7-16f0-41ca-8eaa-035b236bb31d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:12.232556 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007224d4-4b55-4272-b653-9442e73f74f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:12.233213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007224d4-4b55-4272-b653-9442e73f74f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:12.233517 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007224d4-4b55-4272-b653-9442e73f74f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:12.233781 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007224d4-4b55-4272-b653-9442e73f74f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:12.234567 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007224d4-4b55-4272-b653-9442e73f74f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:12.236642 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007224d4-4b55-4272-b653-9442e73f74f8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:12.245155 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1023/2073] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:31:12 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 12:31:12.258550 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97193e5-76b4-44a4-90ff-998078be98a3 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:12.271118 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1051/2074] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:31:12 2022] POST /identity/v3/projects => generated 391 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:31:12.272371 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007224d4-4b55-4272-b653-9442e73f74f8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:12.321088 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:12.325933 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97193e5-76b4-44a4-90ff-998078be98a3 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:12.332241 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1946e7e-c80a-461f-829f-c78ab0de3501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:12.332956 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1946e7e-c80a-461f-829f-c78ab0de3501 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:12.333518 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1946e7e-c80a-461f-829f-c78ab0de3501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:12.333998 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1946e7e-c80a-461f-829f-c78ab0de3501 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:12.334856 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1946e7e-c80a-461f-829f-c78ab0de3501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:12.335978 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007224d4-4b55-4272-b653-9442e73f74f8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:12.336968 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1946e7e-c80a-461f-829f-c78ab0de3501 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:12.340568 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f0f12-4f87-4452-88f9-0deca9cd340b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:12.341174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f0f12-4f87-4452-88f9-0deca9cd340b admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:12.341451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f0f12-4f87-4452-88f9-0deca9cd340b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:12.341722 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f0f12-4f87-4452-88f9-0deca9cd340b admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:12.342372 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f0f12-4f87-4452-88f9-0deca9cd340b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:12.344267 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f0f12-4f87-4452-88f9-0deca9cd340b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:12.360212 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1024/2075] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:31:12 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:31:12.365782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1946e7e-c80a-461f-829f-c78ab0de3501 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:12.373172 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1052/2076] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:31:12 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:31:12.374193 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f0f12-4f87-4452-88f9-0deca9cd340b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:12.379138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1946e7e-c80a-461f-829f-c78ab0de3501 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:12.384613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f0f12-4f87-4452-88f9-0deca9cd340b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:12.385398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38081f8-e1e0-4912-b16f-5251a2b54709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:12.385950 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38081f8-e1e0-4912-b16f-5251a2b54709 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:12.386240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38081f8-e1e0-4912-b16f-5251a2b54709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:12.386507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38081f8-e1e0-4912-b16f-5251a2b54709 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:12.387208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38081f8-e1e0-4912-b16f-5251a2b54709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:12.388826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38081f8-e1e0-4912-b16f-5251a2b54709 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:12.389735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd875144-1e65-4fd6-8d30-262bd4fe69c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:12.390342 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd875144-1e65-4fd6-8d30-262bd4fe69c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:12.390627 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd875144-1e65-4fd6-8d30-262bd4fe69c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:12.390954 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd875144-1e65-4fd6-8d30-262bd4fe69c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:12.391650 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd875144-1e65-4fd6-8d30-262bd4fe69c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:12.393259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd875144-1e65-4fd6-8d30-262bd4fe69c9 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:12.399446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1025/2077] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:31:12.403514 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38081f8-e1e0-4912-b16f-5251a2b54709 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:12.403690 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1053/2078] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 12:31:12.407085 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd875144-1e65-4fd6-8d30-262bd4fe69c9 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:12.414271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38081f8-e1e0-4912-b16f-5251a2b54709 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:12.417017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd875144-1e65-4fd6-8d30-262bd4fe69c9 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:12.418947 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a1a044-3c3c-42cb-b09c-6e20364cbfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:12.419545 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1a044-3c3c-42cb-b09c-6e20364cbfcb admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:12.419830 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1a044-3c3c-42cb-b09c-6e20364cbfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:12.420103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1a044-3c3c-42cb-b09c-6e20364cbfcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:12.420904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a1a044-3c3c-42cb-b09c-6e20364cbfcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:12.421796 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ae075-44ba-4df0-822d-334d89d95572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:12.422382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ae075-44ba-4df0-822d-334d89d95572 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:12.422504 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a1a044-3c3c-42cb-b09c-6e20364cbfcb admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:12.422663 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ae075-44ba-4df0-822d-334d89d95572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:12.422936 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ae075-44ba-4df0-822d-334d89d95572 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:12.423621 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ae075-44ba-4df0-822d-334d89d95572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:12.425915 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ae075-44ba-4df0-822d-334d89d95572 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:12.434281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1026/2079] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 12:31:12.439572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1054/2080] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 12:31:12.439700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a1a044-3c3c-42cb-b09c-6e20364cbfcb admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:12.440939 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ae075-44ba-4df0-822d-334d89d95572 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:12.448928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a1a044-3c3c-42cb-b09c-6e20364cbfcb admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:12.453518 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ae075-44ba-4df0-822d-334d89d95572 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:12.453622 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292ed06-26cd-4ad4-bedc-cc43434e9815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:12.454256 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292ed06-26cd-4ad4-bedc-cc43434e9815 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:12.454539 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292ed06-26cd-4ad4-bedc-cc43434e9815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:12.454814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292ed06-26cd-4ad4-bedc-cc43434e9815 admin admin] PATH_INFO: `/v3/projects/3bcd3d7af528433ca4ced445a5c28ed6/users/aef5498a5f8a41e49b81aa1e1ec2c0e9/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:12.459069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a63df-44db-4bdc-85a6-4298878bf720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:12.459725 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a63df-44db-4bdc-85a6-4298878bf720 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:12.460010 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a63df-44db-4bdc-85a6-4298878bf720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:12.460479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a63df-44db-4bdc-85a6-4298878bf720 admin admin] PATH_INFO: `/v3/projects/27e82164296a4181a64cc9ddc077a612/users/f72a2c39b17b414eab0e740263807a6a/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:12.470402 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292ed06-26cd-4ad4-bedc-cc43434e9815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bcd3d7af528433ca4ced445a5c28ed6, user_id=aef5498a5f8a41e49b81aa1e1ec2c0e9, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:12.473275 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292ed06-26cd-4ad4-bedc-cc43434e9815 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:12.475646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a63df-44db-4bdc-85a6-4298878bf720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27e82164296a4181a64cc9ddc077a612, user_id=f72a2c39b17b414eab0e740263807a6a, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:12.478567 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a63df-44db-4bdc-85a6-4298878bf720 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:12.487985 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1027/2081] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:31:12 2022] PUT /identity/v3/projects/3bcd3d7af528433ca4ced445a5c28ed6/users/aef5498a5f8a41e49b81aa1e1ec2c0e9/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:31:12.492568 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1055/2082] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:31:12 2022] PUT /identity/v3/projects/27e82164296a4181a64cc9ddc077a612/users/f72a2c39b17b414eab0e740263807a6a/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:31:12.498712 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1896cb-1969-4b6f-a9d0-1dc3bcae592d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:12.499000 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1896cb-1969-4b6f-a9d0-1dc3bcae592d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:12.499277 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1896cb-1969-4b6f-a9d0-1dc3bcae592d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:12.502449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab56f6-3c41-4af7-a7e1-f1bf32040ce1 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:12.503261 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab56f6-3c41-4af7-a7e1-f1bf32040ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:12.503538 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab56f6-3c41-4af7-a7e1-f1bf32040ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:12.516382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:12.531633 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-fb1896cb-1969-4b6f-a9d0-1dc3bcae592d None None] MFA Rules not processed for user `aef5498a5f8a41e49b81aa1e1ec2c0e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:12.532912 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fb1896cb-1969-4b6f-a9d0-1dc3bcae592d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:12.534590 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f7ab56f6-3c41-4af7-a7e1-f1bf32040ce1 None None] MFA Rules not processed for user `f72a2c39b17b414eab0e740263807a6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:12.535814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f7ab56f6-3c41-4af7-a7e1-f1bf32040ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:12.623602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1028/2083] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:31:12.634957 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1896cb-1969-4b6f-a9d0-1dc3bcae592d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:12.636393 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fb1896cb-1969-4b6f-a9d0-1dc3bcae592d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:12.655349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1056/2084] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:31:12.668100 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab56f6-3c41-4af7-a7e1-f1bf32040ce1 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:12.669898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f7ab56f6-3c41-4af7-a7e1-f1bf32040ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:12.718711 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1896cb-1969-4b6f-a9d0-1dc3bcae592d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:12.724338 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f696579-4860-4aa1-8895-c2f7b0f55f48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:12.725075 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f696579-4860-4aa1-8895-c2f7b0f55f48 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:12.725416 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f696579-4860-4aa1-8895-c2f7b0f55f48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:12.726207 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f696579-4860-4aa1-8895-c2f7b0f55f48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:12.741254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f696579-4860-4aa1-8895-c2f7b0f55f48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:12.743209 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f696579-4860-4aa1-8895-c2f7b0f55f48 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:12.753791 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab56f6-3c41-4af7-a7e1-f1bf32040ce1 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:12.758649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9a703-d5a1-4a71-9218-d003e7a347ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:12.759881 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9a703-d5a1-4a71-9218-d003e7a347ff service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:12.759881 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9a703-d5a1-4a71-9218-d003e7a347ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:12.760073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9a703-d5a1-4a71-9218-d003e7a347ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:12.766687 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1029/2085] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:31:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:31:12.770538 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b9a703-d5a1-4a71-9218-d003e7a347ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:12.785648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b9a703-d5a1-4a71-9218-d003e7a347ff service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:12.810507 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1057/2086] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:31:12 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:31:22.749744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=81,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:22.800954 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=55 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:27.082119 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f696579-4860-4aa1-8895-c2f7b0f55f48 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:27.146483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f696579-4860-4aa1-8895-c2f7b0f55f48 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:27.151021 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023eef0-686a-452b-99f2-2a6152667e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:27.151659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023eef0-686a-452b-99f2-2a6152667e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:27.151938 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023eef0-686a-452b-99f2-2a6152667e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:27.152208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023eef0-686a-452b-99f2-2a6152667e7b admin admin] PATH_INFO: `/v3/users/aef5498a5f8a41e49b81aa1e1ec2c0e9` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:27.153422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023eef0-686a-452b-99f2-2a6152667e7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=aef5498a5f8a41e49b81aa1e1ec2c0e9)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:27.155349 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023eef0-686a-452b-99f2-2a6152667e7b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:27.205368 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a023eef0-686a-452b-99f2-2a6152667e7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aef5498a5f8a41e49b81aa1e1ec2c0e9'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:27.212728 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a023eef0-686a-452b-99f2-2a6152667e7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aef5498a5f8a41e49b81aa1e1ec2c0e9'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:27.227901 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a023eef0-686a-452b-99f2-2a6152667e7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aef5498a5f8a41e49b81aa1e1ec2c0e9'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:27.228745 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a023eef0-686a-452b-99f2-2a6152667e7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aef5498a5f8a41e49b81aa1e1ec2c0e9'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:27.229596 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a023eef0-686a-452b-99f2-2a6152667e7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aef5498a5f8a41e49b81aa1e1ec2c0e9'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:27.254260 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1030/2087] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:31:27 2022] DELETE /identity/v3/users/aef5498a5f8a41e49b81aa1e1ec2c0e9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:27.258558 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bd842-d380-4013-b690-e6037d147b7c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:27.259371 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bd842-d380-4013-b690-e6037d147b7c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:27.259647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bd842-d380-4013-b690-e6037d147b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:27.280164 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-485bd842-d380-4013-b690-e6037d147b7c None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:27.281393 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-485bd842-d380-4013-b690-e6037d147b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:27.352108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1058/2088] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:31:27.366350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023eef0-686a-452b-99f2-2a6152667e7b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:27.367772 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a023eef0-686a-452b-99f2-2a6152667e7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:27.432585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023eef0-686a-452b-99f2-2a6152667e7b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:27.436360 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd05f2-9a3e-4798-962d-652a40f91a85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:27.436970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd05f2-9a3e-4798-962d-652a40f91a85 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:27.437246 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd05f2-9a3e-4798-962d-652a40f91a85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:27.437515 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd05f2-9a3e-4798-962d-652a40f91a85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:27.447529 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd05f2-9a3e-4798-962d-652a40f91a85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:27.449000 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd05f2-9a3e-4798-962d-652a40f91a85 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:27.459112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1031/2089] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:31:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:31:27.756919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bd842-d380-4013-b690-e6037d147b7c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:27.758316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-485bd842-d380-4013-b690-e6037d147b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:27.773266 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bd842-d380-4013-b690-e6037d147b7c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:27.778157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b97e9e-6400-4249-85f0-24c0d42c0851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:27.778812 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b97e9e-6400-4249-85f0-24c0d42c0851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:27.779085 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b97e9e-6400-4249-85f0-24c0d42c0851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:27.779413 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b97e9e-6400-4249-85f0-24c0d42c0851 admin admin] PATH_INFO: `/v3/projects/3bcd3d7af528433ca4ced445a5c28ed6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:27.780673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b97e9e-6400-4249-85f0-24c0d42c0851 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bcd3d7af528433ca4ced445a5c28ed6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:27.782784 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b97e9e-6400-4249-85f0-24c0d42c0851 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:27.828720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c1b97e9e-6400-4249-85f0-24c0d42c0851 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bcd3d7af528433ca4ced445a5c28ed6', 'request_id': 'req-c1b97e9e-6400-4249-85f0-24c0d42c0851'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:27.834590 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c1b97e9e-6400-4249-85f0-24c0d42c0851 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bcd3d7af528433ca4ced445a5c28ed6', 'request_id': 'req-c1b97e9e-6400-4249-85f0-24c0d42c0851'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:27.841850 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c1b97e9e-6400-4249-85f0-24c0d42c0851 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bcd3d7af528433ca4ced445a5c28ed6', 'request_id': 'req-c1b97e9e-6400-4249-85f0-24c0d42c0851'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:27.853438 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c1b97e9e-6400-4249-85f0-24c0d42c0851 admin admin] The token cache is being invalidate because project 3bcd3d7af528433ca4ced445a5c28ed6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:31:27.853723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c1b97e9e-6400-4249-85f0-24c0d42c0851 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:27.854338 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c1b97e9e-6400-4249-85f0-24c0d42c0851 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:27.855824 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1059/2090] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:31:27 2022] DELETE /identity/v3/projects/3bcd3d7af528433ca4ced445a5c28ed6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:27.860785 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672b610-672a-4a7d-b1a4-edb5f2666a02 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:27.861073 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672b610-672a-4a7d-b1a4-edb5f2666a02 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:27.861397 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672b610-672a-4a7d-b1a4-edb5f2666a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:27.880508 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-a672b610-672a-4a7d-b1a4-edb5f2666a02 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:27.881701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a672b610-672a-4a7d-b1a4-edb5f2666a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:27.898270 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:27.941662 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1032/2091] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:27.950729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba1e5d-8ad2-4c1e-822c-bfb3b7567d86 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:27.951111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba1e5d-8ad2-4c1e-822c-bfb3b7567d86 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:27.951459 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba1e5d-8ad2-4c1e-822c-bfb3b7567d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:27.971249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-65ba1e5d-8ad2-4c1e-822c-bfb3b7567d86 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:27.972481 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-65ba1e5d-8ad2-4c1e-822c-bfb3b7567d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:27.980170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1060/2092] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:27.984249 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672b610-672a-4a7d-b1a4-edb5f2666a02 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:27.994941 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672b610-672a-4a7d-b1a4-edb5f2666a02 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:27.999463 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ac59a-a077-4ddf-91f2-5575df225483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:28.000070 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ac59a-a077-4ddf-91f2-5575df225483 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:28.000365 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ac59a-a077-4ddf-91f2-5575df225483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:28.000634 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ac59a-a077-4ddf-91f2-5575df225483 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:28.001321 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ac59a-a077-4ddf-91f2-5575df225483 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:31:28.001623 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ac59a-a077-4ddf-91f2-5575df225483 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:28.003202 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ac59a-a077-4ddf-91f2-5575df225483 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:28.014684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1033/2093] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:31:28.019014 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba1e5d-8ad2-4c1e-822c-bfb3b7567d86 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:28.028372 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba1e5d-8ad2-4c1e-822c-bfb3b7567d86 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:28.033097 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4699e-b32b-41d6-bd43-74760017abcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:28.034231 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4699e-b32b-41d6-bd43-74760017abcc admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:28.034508 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4699e-b32b-41d6-bd43-74760017abcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:28.034810 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4699e-b32b-41d6-bd43-74760017abcc admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:28.035487 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4699e-b32b-41d6-bd43-74760017abcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:28.037424 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4699e-b32b-41d6-bd43-74760017abcc admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:28.065820 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1061/2094] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:31:28 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:31:28.069877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ac59a-a077-4ddf-91f2-5575df225483 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:28.079575 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ac59a-a077-4ddf-91f2-5575df225483 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:28.134034 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36138b13-0f4a-4223-bf57-bfa7ac1d0a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:28.134651 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36138b13-0f4a-4223-bf57-bfa7ac1d0a0b admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:28.134887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36138b13-0f4a-4223-bf57-bfa7ac1d0a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:28.135155 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36138b13-0f4a-4223-bf57-bfa7ac1d0a0b admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:28.135802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36138b13-0f4a-4223-bf57-bfa7ac1d0a0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:28.137750 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36138b13-0f4a-4223-bf57-bfa7ac1d0a0b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:28.158012 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1034/2095] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:31:28 2022] POST /identity/v3/users => generated 282 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 12:31:28.162606 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4699e-b32b-41d6-bd43-74760017abcc admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:28.172842 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4699e-b32b-41d6-bd43-74760017abcc admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:28.177457 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3d420-6e4b-4526-9f42-d4cd4a1f2e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:28.178056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3d420-6e4b-4526-9f42-d4cd4a1f2e20 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:28.178337 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3d420-6e4b-4526-9f42-d4cd4a1f2e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:28.178613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3d420-6e4b-4526-9f42-d4cd4a1f2e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:28.179334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3d420-6e4b-4526-9f42-d4cd4a1f2e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:28.180951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3d420-6e4b-4526-9f42-d4cd4a1f2e20 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:28.191170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1062/2096] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:31:28.194814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36138b13-0f4a-4223-bf57-bfa7ac1d0a0b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:28.204475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36138b13-0f4a-4223-bf57-bfa7ac1d0a0b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:28.209113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5efb70-5184-4773-a875-6edbbb18cccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:28.209684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5efb70-5184-4773-a875-6edbbb18cccb admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:28.209959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5efb70-5184-4773-a875-6edbbb18cccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:28.210227 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5efb70-5184-4773-a875-6edbbb18cccb admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:28.210902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5efb70-5184-4773-a875-6edbbb18cccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:28.212507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5efb70-5184-4773-a875-6edbbb18cccb admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:28.223223 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1035/2097] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:31:28.227204 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3d420-6e4b-4526-9f42-d4cd4a1f2e20 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:28.237838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3d420-6e4b-4526-9f42-d4cd4a1f2e20 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:28.242756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487757dd-6180-4cba-adb3-b73b05c729a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:28.243391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487757dd-6180-4cba-adb3-b73b05c729a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:28.243667 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487757dd-6180-4cba-adb3-b73b05c729a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:28.243938 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487757dd-6180-4cba-adb3-b73b05c729a8 admin admin] PATH_INFO: `/v3/projects/d63841b5dd4841d99ea3c8f456308095/users/7a31153e522c457dad7154a2965db807/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:28.258625 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487757dd-6180-4cba-adb3-b73b05c729a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d63841b5dd4841d99ea3c8f456308095, user_id=7a31153e522c457dad7154a2965db807, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:28.261301 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487757dd-6180-4cba-adb3-b73b05c729a8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:28.275654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1063/2098] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:31:28 2022] PUT /identity/v3/projects/d63841b5dd4841d99ea3c8f456308095/users/7a31153e522c457dad7154a2965db807/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:31:28.286495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2c9bd-d672-46cf-bb73-ba45a5f5803c None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:28.286791 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2c9bd-d672-46cf-bb73-ba45a5f5803c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:28.287066 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2c9bd-d672-46cf-bb73-ba45a5f5803c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:28.315670 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-d4d2c9bd-d672-46cf-bb73-ba45a5f5803c None None] MFA Rules not processed for user `7a31153e522c457dad7154a2965db807`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:28.316888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d4d2c9bd-d672-46cf-bb73-ba45a5f5803c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:28.379692 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1036/2099] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:28.390724 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487757dd-6180-4cba-adb3-b73b05c729a8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:28.392405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-487757dd-6180-4cba-adb3-b73b05c729a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:28.413580 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:28.452468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487757dd-6180-4cba-adb3-b73b05c729a8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:28.456134 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df34bf-188e-45bc-968c-4f7d1c6d3d89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:28.456779 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df34bf-188e-45bc-968c-4f7d1c6d3d89 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:28.457057 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df34bf-188e-45bc-968c-4f7d1c6d3d89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:28.457326 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df34bf-188e-45bc-968c-4f7d1c6d3d89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:28.466872 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df34bf-188e-45bc-968c-4f7d1c6d3d89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:28.468327 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df34bf-188e-45bc-968c-4f7d1c6d3d89 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:28.477570 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1064/2100] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:31:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:31:38.376202 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:38.473302 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=17 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:41.408106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d2c9bd-d672-46cf-bb73-ba45a5f5803c None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:41.409637 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d4d2c9bd-d672-46cf-bb73-ba45a5f5803c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:41.540053 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d2c9bd-d672-46cf-bb73-ba45a5f5803c None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:41.547863 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce382ce5-8287-4bb9-b28d-e33eded2bbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:41.549742 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce382ce5-8287-4bb9-b28d-e33eded2bbcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:41.550032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce382ce5-8287-4bb9-b28d-e33eded2bbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:41.550309 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce382ce5-8287-4bb9-b28d-e33eded2bbcb admin admin] PATH_INFO: `/v3/users/6e424addc3e3403991b16ca321239d00` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:41.553289 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce382ce5-8287-4bb9-b28d-e33eded2bbcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e424addc3e3403991b16ca321239d00)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:41.555530 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce382ce5-8287-4bb9-b28d-e33eded2bbcb admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:41.617127 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ce382ce5-8287-4bb9-b28d-e33eded2bbcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e424addc3e3403991b16ca321239d00'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:41.623772 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ce382ce5-8287-4bb9-b28d-e33eded2bbcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e424addc3e3403991b16ca321239d00'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:41.635830 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ce382ce5-8287-4bb9-b28d-e33eded2bbcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e424addc3e3403991b16ca321239d00'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:41.636534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ce382ce5-8287-4bb9-b28d-e33eded2bbcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e424addc3e3403991b16ca321239d00'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:41.637150 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ce382ce5-8287-4bb9-b28d-e33eded2bbcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e424addc3e3403991b16ca321239d00'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:41.649987 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1037/2101] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:31:41 2022] DELETE /identity/v3/users/6e424addc3e3403991b16ca321239d00 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:41.654531 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947d7fc-9f45-4f8f-8c27-1c219c0031ca None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:41.654999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947d7fc-9f45-4f8f-8c27-1c219c0031ca None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:41.655222 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947d7fc-9f45-4f8f-8c27-1c219c0031ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:41.674703 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5947d7fc-9f45-4f8f-8c27-1c219c0031ca None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:41.675875 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5947d7fc-9f45-4f8f-8c27-1c219c0031ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:41.733184 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1065/2102] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:41.743559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce382ce5-8287-4bb9-b28d-e33eded2bbcb admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:41.745024 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ce382ce5-8287-4bb9-b28d-e33eded2bbcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:41.801412 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce382ce5-8287-4bb9-b28d-e33eded2bbcb admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:41.805060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba66d4c-065a-45e8-a45b-e59df51d09b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:41.805688 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba66d4c-065a-45e8-a45b-e59df51d09b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:41.805962 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba66d4c-065a-45e8-a45b-e59df51d09b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:41.806231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba66d4c-065a-45e8-a45b-e59df51d09b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:41.815167 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba66d4c-065a-45e8-a45b-e59df51d09b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:41.816674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba66d4c-065a-45e8-a45b-e59df51d09b7 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:41.826225 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1038/2103] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:31:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:31:42.125363 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947d7fc-9f45-4f8f-8c27-1c219c0031ca None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:42.127006 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5947d7fc-9f45-4f8f-8c27-1c219c0031ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:42.142860 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947d7fc-9f45-4f8f-8c27-1c219c0031ca None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:42.147843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a089940-ca59-421f-8304-492b3c67e401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:42.149407 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a089940-ca59-421f-8304-492b3c67e401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:42.150045 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a089940-ca59-421f-8304-492b3c67e401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:42.150324 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a089940-ca59-421f-8304-492b3c67e401 admin admin] PATH_INFO: `/v3/projects/c31ed4d577414dd0be841b28e1795bae` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:42.151507 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a089940-ca59-421f-8304-492b3c67e401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c31ed4d577414dd0be841b28e1795bae)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:42.153842 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a089940-ca59-421f-8304-492b3c67e401 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:42.208027 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2a089940-ca59-421f-8304-492b3c67e401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c31ed4d577414dd0be841b28e1795bae', 'request_id': 'req-2a089940-ca59-421f-8304-492b3c67e401'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:42.214021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2a089940-ca59-421f-8304-492b3c67e401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c31ed4d577414dd0be841b28e1795bae', 'request_id': 'req-2a089940-ca59-421f-8304-492b3c67e401'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:42.221197 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2a089940-ca59-421f-8304-492b3c67e401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c31ed4d577414dd0be841b28e1795bae', 'request_id': 'req-2a089940-ca59-421f-8304-492b3c67e401'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:42.232208 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2a089940-ca59-421f-8304-492b3c67e401 admin admin] The token cache is being invalidate because project c31ed4d577414dd0be841b28e1795bae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:31:42.232529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2a089940-ca59-421f-8304-492b3c67e401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:42.233160 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2a089940-ca59-421f-8304-492b3c67e401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:42.234740 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1066/2104] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:31:42 2022] DELETE /identity/v3/projects/c31ed4d577414dd0be841b28e1795bae => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:42.239902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d9200-2438-4d69-95c5-b65a30cf0e66 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:42.240476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d9200-2438-4d69-95c5-b65a30cf0e66 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:42.240751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d9200-2438-4d69-95c5-b65a30cf0e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:42.260993 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-524d9200-2438-4d69-95c5-b65a30cf0e66 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:42.262193 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-524d9200-2438-4d69-95c5-b65a30cf0e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:42.279361 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:42.322136 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1039/2105] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:42.330909 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d878c3-0f73-4897-a02f-50863ba76c5e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:42.331217 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d878c3-0f73-4897-a02f-50863ba76c5e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:42.331531 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d878c3-0f73-4897-a02f-50863ba76c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:42.351711 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-74d878c3-0f73-4897-a02f-50863ba76c5e None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:42.352951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-74d878c3-0f73-4897-a02f-50863ba76c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:42.361472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1067/2106] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:42.365792 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d9200-2438-4d69-95c5-b65a30cf0e66 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:42.375324 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d9200-2438-4d69-95c5-b65a30cf0e66 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:42.379996 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f09c5-c725-484c-84f7-79a99c5fb696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:42.380632 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f09c5-c725-484c-84f7-79a99c5fb696 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:42.380912 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f09c5-c725-484c-84f7-79a99c5fb696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:42.381182 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f09c5-c725-484c-84f7-79a99c5fb696 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:42.381868 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981f09c5-c725-484c-84f7-79a99c5fb696 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:31:42.382170 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981f09c5-c725-484c-84f7-79a99c5fb696 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:42.383753 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981f09c5-c725-484c-84f7-79a99c5fb696 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:42.395448 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1040/2107] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 12:31:42.399668 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d878c3-0f73-4897-a02f-50863ba76c5e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:42.409755 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d878c3-0f73-4897-a02f-50863ba76c5e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:42.414601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58ff5e-63e0-4eb4-9b82-7348e4dd0926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:42.415201 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58ff5e-63e0-4eb4-9b82-7348e4dd0926 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:42.415486 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58ff5e-63e0-4eb4-9b82-7348e4dd0926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:42.415759 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58ff5e-63e0-4eb4-9b82-7348e4dd0926 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:42.416455 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d58ff5e-63e0-4eb4-9b82-7348e4dd0926 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:42.418348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d58ff5e-63e0-4eb4-9b82-7348e4dd0926 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:42.445468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1068/2108] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:31:42 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:31:42.449509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f09c5-c725-484c-84f7-79a99c5fb696 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:42.460822 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f09c5-c725-484c-84f7-79a99c5fb696 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:42.528889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90f04c-2d7e-4800-b64b-ce49e766452a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:42.529585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90f04c-2d7e-4800-b64b-ce49e766452a admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:42.529978 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90f04c-2d7e-4800-b64b-ce49e766452a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:42.530284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90f04c-2d7e-4800-b64b-ce49e766452a admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:42.530943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd90f04c-2d7e-4800-b64b-ce49e766452a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:42.534514 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd90f04c-2d7e-4800-b64b-ce49e766452a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:42.558550 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1041/2109] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:31:42 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:31:42.562854 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58ff5e-63e0-4eb4-9b82-7348e4dd0926 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:42.573427 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58ff5e-63e0-4eb4-9b82-7348e4dd0926 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:42.578077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3b545-fdef-4cbb-9f3e-52709aa36332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:42.578674 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3b545-fdef-4cbb-9f3e-52709aa36332 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:42.578961 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3b545-fdef-4cbb-9f3e-52709aa36332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:42.579235 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3b545-fdef-4cbb-9f3e-52709aa36332 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:42.580997 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3b545-fdef-4cbb-9f3e-52709aa36332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:42.582052 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3b545-fdef-4cbb-9f3e-52709aa36332 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:42.592278 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1069/2110] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:31:42.596285 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90f04c-2d7e-4800-b64b-ce49e766452a admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:42.610140 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90f04c-2d7e-4800-b64b-ce49e766452a admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:42.616866 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02548f3f-699f-43e5-ac5c-1b9a6614d436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:42.618047 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02548f3f-699f-43e5-ac5c-1b9a6614d436 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:42.618047 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02548f3f-699f-43e5-ac5c-1b9a6614d436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:42.618237 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02548f3f-699f-43e5-ac5c-1b9a6614d436 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:42.619076 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02548f3f-699f-43e5-ac5c-1b9a6614d436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:42.620950 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02548f3f-699f-43e5-ac5c-1b9a6614d436 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:42.637796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1042/2111] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:31:42.640433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3b545-fdef-4cbb-9f3e-52709aa36332 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:42.653615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3b545-fdef-4cbb-9f3e-52709aa36332 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:42.660613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dac0c3-9f34-4ded-b503-dff52cc477f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:42.662305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dac0c3-9f34-4ded-b503-dff52cc477f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:42.662768 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dac0c3-9f34-4ded-b503-dff52cc477f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:42.663588 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dac0c3-9f34-4ded-b503-dff52cc477f1 admin admin] PATH_INFO: `/v3/projects/c1cbbde8a7a342d6a74e14d9010e91e0/users/d36ce80ae6a942f1866cd2b0a5fe4c01/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:42.682739 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dac0c3-9f34-4ded-b503-dff52cc477f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1cbbde8a7a342d6a74e14d9010e91e0, user_id=d36ce80ae6a942f1866cd2b0a5fe4c01, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:42.685582 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dac0c3-9f34-4ded-b503-dff52cc477f1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:42.702655 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1070/2112] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:31:42 2022] PUT /identity/v3/projects/c1cbbde8a7a342d6a74e14d9010e91e0/users/d36ce80ae6a942f1866cd2b0a5fe4c01/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:31:42.714208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595bd724-f0e4-4fe9-9e91-97e53c75f2d5 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:42.714703 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595bd724-f0e4-4fe9-9e91-97e53c75f2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:42.715036 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595bd724-f0e4-4fe9-9e91-97e53c75f2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:42.751356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-595bd724-f0e4-4fe9-9e91-97e53c75f2d5 None None] MFA Rules not processed for user `d36ce80ae6a942f1866cd2b0a5fe4c01`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:42.752643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-595bd724-f0e4-4fe9-9e91-97e53c75f2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:42.835679 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1043/2113] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:42.847647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dac0c3-9f34-4ded-b503-dff52cc477f1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:42.849488 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d2dac0c3-9f34-4ded-b503-dff52cc477f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:42.871405 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:42.911300 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dac0c3-9f34-4ded-b503-dff52cc477f1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:42.915473 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13992e2-c693-41ca-ae33-63a24da35759 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:42.916353 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13992e2-c693-41ca-ae33-63a24da35759 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:42.916647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13992e2-c693-41ca-ae33-63a24da35759 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:42.916923 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13992e2-c693-41ca-ae33-63a24da35759 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:42.926609 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13992e2-c693-41ca-ae33-63a24da35759 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:42.928046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13992e2-c693-41ca-ae33-63a24da35759 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:42.937115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1071/2114] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:31:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:31:45.361686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa40c5-858f-47ba-84f1-72dab02e8875 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:45.362210 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa40c5-858f-47ba-84f1-72dab02e8875 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:45.362210 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa40c5-858f-47ba-84f1-72dab02e8875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:45.381176 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-33aa40c5-858f-47ba-84f1-72dab02e8875 None None] MFA Rules not processed for user `d36ce80ae6a942f1866cd2b0a5fe4c01`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:45.382373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-33aa40c5-858f-47ba-84f1-72dab02e8875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:45.389045 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1044/2115] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:50.177417 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13992e2-c693-41ca-ae33-63a24da35759 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:50.179861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b13992e2-c693-41ca-ae33-63a24da35759 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:50.274681 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13992e2-c693-41ca-ae33-63a24da35759 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:50.274784 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147fb1a-22e6-4185-8449-3e100e41cf21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:50.276106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147fb1a-22e6-4185-8449-3e100e41cf21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:50.278901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147fb1a-22e6-4185-8449-3e100e41cf21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:50.279441 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147fb1a-22e6-4185-8449-3e100e41cf21 admin admin] PATH_INFO: `/v3/users/f72a2c39b17b414eab0e740263807a6a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:50.281070 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c147fb1a-22e6-4185-8449-3e100e41cf21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f72a2c39b17b414eab0e740263807a6a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:50.283281 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c147fb1a-22e6-4185-8449-3e100e41cf21 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:50.346298 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c147fb1a-22e6-4185-8449-3e100e41cf21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f72a2c39b17b414eab0e740263807a6a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:50.363443 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c147fb1a-22e6-4185-8449-3e100e41cf21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f72a2c39b17b414eab0e740263807a6a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:50.364212 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c147fb1a-22e6-4185-8449-3e100e41cf21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f72a2c39b17b414eab0e740263807a6a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:50.364887 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c147fb1a-22e6-4185-8449-3e100e41cf21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f72a2c39b17b414eab0e740263807a6a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:50.372277 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c147fb1a-22e6-4185-8449-3e100e41cf21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f72a2c39b17b414eab0e740263807a6a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:50.386080 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1072/2116] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:31:50 2022] DELETE /identity/v3/users/f72a2c39b17b414eab0e740263807a6a => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:50.390678 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ae9be-eaf8-47e4-af71-4d672359599c None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:50.391140 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ae9be-eaf8-47e4-af71-4d672359599c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:50.391252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ae9be-eaf8-47e4-af71-4d672359599c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:50.410513 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-b71ae9be-eaf8-47e4-af71-4d672359599c None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:50.411689 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b71ae9be-eaf8-47e4-af71-4d672359599c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:50.417387 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:50.481408 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1045/2117] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:50.490682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147fb1a-22e6-4185-8449-3e100e41cf21 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:50.491864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c147fb1a-22e6-4185-8449-3e100e41cf21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:50.542413 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:50.557448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147fb1a-22e6-4185-8449-3e100e41cf21 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:50.561062 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c57cf-66bf-42e5-a284-bcb98532ca7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:50.561677 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c57cf-66bf-42e5-a284-bcb98532ca7d service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:50.561984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c57cf-66bf-42e5-a284-bcb98532ca7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:50.562259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c57cf-66bf-42e5-a284-bcb98532ca7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:50.571876 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c57cf-66bf-42e5-a284-bcb98532ca7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:50.573497 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c57cf-66bf-42e5-a284-bcb98532ca7d service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:50.584798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1073/2118] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:31:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:31:51.078782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ae9be-eaf8-47e4-af71-4d672359599c None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:51.080823 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b71ae9be-eaf8-47e4-af71-4d672359599c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:51.093949 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ae9be-eaf8-47e4-af71-4d672359599c None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:51.114185 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582374d7-3b52-4b0f-97bf-3cb52493428f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:51.114887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582374d7-3b52-4b0f-97bf-3cb52493428f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:51.116698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582374d7-3b52-4b0f-97bf-3cb52493428f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:51.116979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582374d7-3b52-4b0f-97bf-3cb52493428f admin admin] PATH_INFO: `/v3/projects/27e82164296a4181a64cc9ddc077a612` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:51.118348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582374d7-3b52-4b0f-97bf-3cb52493428f admin admin] RBAC: Authorizing `identity:delete_project(project_id=27e82164296a4181a64cc9ddc077a612)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:51.120289 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582374d7-3b52-4b0f-97bf-3cb52493428f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:51.177008 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-582374d7-3b52-4b0f-97bf-3cb52493428f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27e82164296a4181a64cc9ddc077a612', 'request_id': 'req-582374d7-3b52-4b0f-97bf-3cb52493428f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:51.182785 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-582374d7-3b52-4b0f-97bf-3cb52493428f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27e82164296a4181a64cc9ddc077a612', 'request_id': 'req-582374d7-3b52-4b0f-97bf-3cb52493428f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:51.191236 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-582374d7-3b52-4b0f-97bf-3cb52493428f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27e82164296a4181a64cc9ddc077a612', 'request_id': 'req-582374d7-3b52-4b0f-97bf-3cb52493428f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:51.203690 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-582374d7-3b52-4b0f-97bf-3cb52493428f admin admin] The token cache is being invalidate because project 27e82164296a4181a64cc9ddc077a612 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:31:51.203979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-582374d7-3b52-4b0f-97bf-3cb52493428f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:51.204681 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-582374d7-3b52-4b0f-97bf-3cb52493428f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:51.206324 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1046/2119] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:31:51 2022] DELETE /identity/v3/projects/27e82164296a4181a64cc9ddc077a612 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:54.054952 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c57cf-66bf-42e5-a284-bcb98532ca7d service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:54.056436 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9c9c57cf-66bf-42e5-a284-bcb98532ca7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:54.070613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c57cf-66bf-42e5-a284-bcb98532ca7d service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:54.134579 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720870d6-2644-4632-9ee8-9193df490a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:54.135948 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720870d6-2644-4632-9ee8-9193df490a72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:54.136222 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720870d6-2644-4632-9ee8-9193df490a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:54.136530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720870d6-2644-4632-9ee8-9193df490a72 admin admin] PATH_INFO: `/v3/users/79c2d686d2034765ac50f7ad75295197` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:54.137764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720870d6-2644-4632-9ee8-9193df490a72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79c2d686d2034765ac50f7ad75295197)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:54.139710 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720870d6-2644-4632-9ee8-9193df490a72 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:54.192919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-720870d6-2644-4632-9ee8-9193df490a72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79c2d686d2034765ac50f7ad75295197'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:54.205577 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-720870d6-2644-4632-9ee8-9193df490a72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79c2d686d2034765ac50f7ad75295197'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:54.206309 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-720870d6-2644-4632-9ee8-9193df490a72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79c2d686d2034765ac50f7ad75295197'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:54.206908 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-720870d6-2644-4632-9ee8-9193df490a72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79c2d686d2034765ac50f7ad75295197'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:54.215046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-720870d6-2644-4632-9ee8-9193df490a72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79c2d686d2034765ac50f7ad75295197'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:54.227604 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1074/2120] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:31:54 2022] DELETE /identity/v3/users/79c2d686d2034765ac50f7ad75295197 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:31:54.402845 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582374d7-3b52-4b0f-97bf-3cb52493428f admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:54.404283 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-582374d7-3b52-4b0f-97bf-3cb52493428f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:54.419465 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582374d7-3b52-4b0f-97bf-3cb52493428f admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:54.477875 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112a674-8db7-4e03-b904-5bbe60ab9047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:54.478616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112a674-8db7-4e03-b904-5bbe60ab9047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:54.478886 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112a674-8db7-4e03-b904-5bbe60ab9047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:54.479046 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112a674-8db7-4e03-b904-5bbe60ab9047 admin admin] PATH_INFO: `/v3/projects/6111202e81324d069fae641cc1606e77` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:54.480215 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112a674-8db7-4e03-b904-5bbe60ab9047 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6111202e81324d069fae641cc1606e77)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:54.482162 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112a674-8db7-4e03-b904-5bbe60ab9047 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:54.538499 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:54.547426 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6112a674-8db7-4e03-b904-5bbe60ab9047 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6111202e81324d069fae641cc1606e77', 'request_id': 'req-6112a674-8db7-4e03-b904-5bbe60ab9047'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:54.559595 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6112a674-8db7-4e03-b904-5bbe60ab9047 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6111202e81324d069fae641cc1606e77', 'request_id': 'req-6112a674-8db7-4e03-b904-5bbe60ab9047'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:54.567526 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6112a674-8db7-4e03-b904-5bbe60ab9047 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6111202e81324d069fae641cc1606e77', 'request_id': 'req-6112a674-8db7-4e03-b904-5bbe60ab9047'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:54.579804 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6112a674-8db7-4e03-b904-5bbe60ab9047 admin admin] The token cache is being invalidate because project 6111202e81324d069fae641cc1606e77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:31:54.580161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6112a674-8db7-4e03-b904-5bbe60ab9047 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:54.580930 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6112a674-8db7-4e03-b904-5bbe60ab9047 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:31:54.582538 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1047/2121] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:31:54 2022] DELETE /identity/v3/projects/6111202e81324d069fae641cc1606e77 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:31:54.588490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbeed75-0e32-40dd-aeda-8e7d81345c4b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:54.588805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbeed75-0e32-40dd-aeda-8e7d81345c4b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:54.589085 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbeed75-0e32-40dd-aeda-8e7d81345c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:54.609134 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1dbeed75-0e32-40dd-aeda-8e7d81345c4b None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:54.610396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1dbeed75-0e32-40dd-aeda-8e7d81345c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:54.678642 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1075/2122] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:54.686906 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e7660-3003-499d-acb1-d3fd4a1994ee None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:54.687221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e7660-3003-499d-acb1-d3fd4a1994ee None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:54.687457 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e7660-3003-499d-acb1-d3fd4a1994ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:54.708290 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-d87e7660-3003-499d-acb1-d3fd4a1994ee None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:54.709531 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d87e7660-3003-499d-acb1-d3fd4a1994ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:54.725852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1048/2123] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:31:54.730818 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbeed75-0e32-40dd-aeda-8e7d81345c4b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:54.742246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:31:54.747667 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbeed75-0e32-40dd-aeda-8e7d81345c4b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:54.754635 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd2774-9cfc-4aa8-a409-346ae3664588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:54.755326 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd2774-9cfc-4aa8-a409-346ae3664588 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:54.755614 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd2774-9cfc-4aa8-a409-346ae3664588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:54.755894 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd2774-9cfc-4aa8-a409-346ae3664588 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:54.756630 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd2774-9cfc-4aa8-a409-346ae3664588 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:31:54.756938 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd2774-9cfc-4aa8-a409-346ae3664588 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:54.758541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd2774-9cfc-4aa8-a409-346ae3664588 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:54.776795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1076/2124] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:31:54.778964 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e7660-3003-499d-acb1-d3fd4a1994ee None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:54.787629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e7660-3003-499d-acb1-d3fd4a1994ee None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:54.793028 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d61d9-9bc1-42f3-9f81-cad5652db26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:54.793644 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d61d9-9bc1-42f3-9f81-cad5652db26f admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:54.794385 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d61d9-9bc1-42f3-9f81-cad5652db26f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:54.794385 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d61d9-9bc1-42f3-9f81-cad5652db26f admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:54.794841 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d61d9-9bc1-42f3-9f81-cad5652db26f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:54.797491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d61d9-9bc1-42f3-9f81-cad5652db26f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:54.829491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1049/2125] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:31:54 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:31:54.830716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd2774-9cfc-4aa8-a409-346ae3664588 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:54.840812 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd2774-9cfc-4aa8-a409-346ae3664588 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:54.898298 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e0628-b0c0-4c77-88af-272cb85232a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:54.898913 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e0628-b0c0-4c77-88af-272cb85232a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:54.899194 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e0628-b0c0-4c77-88af-272cb85232a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:54.899469 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e0628-b0c0-4c77-88af-272cb85232a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:54.900132 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e0628-b0c0-4c77-88af-272cb85232a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:54.902381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e0628-b0c0-4c77-88af-272cb85232a9 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:54.924079 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1077/2126] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:31:54 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 12:31:54.928314 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d61d9-9bc1-42f3-9f81-cad5652db26f admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:54.938554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d61d9-9bc1-42f3-9f81-cad5652db26f admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:54.943137 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c3e79-39fd-4b72-abf0-c5894f6906b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:54.943730 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c3e79-39fd-4b72-abf0-c5894f6906b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:54.944009 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c3e79-39fd-4b72-abf0-c5894f6906b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:54.944281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c3e79-39fd-4b72-abf0-c5894f6906b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:54.944995 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8c3e79-39fd-4b72-abf0-c5894f6906b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:54.946607 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8c3e79-39fd-4b72-abf0-c5894f6906b0 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:54.966935 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1050/2127] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:31:54.971459 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e0628-b0c0-4c77-88af-272cb85232a9 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:54.986033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e0628-b0c0-4c77-88af-272cb85232a9 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:54.994207 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782bab9-f954-4e76-8ac2-76503e7c5899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:54.995389 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782bab9-f954-4e76-8ac2-76503e7c5899 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:54.995389 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782bab9-f954-4e76-8ac2-76503e7c5899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:54.995574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782bab9-f954-4e76-8ac2-76503e7c5899 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:54.996473 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782bab9-f954-4e76-8ac2-76503e7c5899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:54.998168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782bab9-f954-4e76-8ac2-76503e7c5899 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:55.011162 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1078/2128] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:31:55.015482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c3e79-39fd-4b72-abf0-c5894f6906b0 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:55.025509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c3e79-39fd-4b72-abf0-c5894f6906b0 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:55.030100 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03996a8f-1713-4bb3-ae3e-00754bd20926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:55.031370 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03996a8f-1713-4bb3-ae3e-00754bd20926 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:55.031660 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03996a8f-1713-4bb3-ae3e-00754bd20926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:55.031931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03996a8f-1713-4bb3-ae3e-00754bd20926 admin admin] PATH_INFO: `/v3/projects/4931a52d9cda40b89d15804b69b5a2d7/users/6844f315435f48f68488a2f0459bf155/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:55.046704 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03996a8f-1713-4bb3-ae3e-00754bd20926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4931a52d9cda40b89d15804b69b5a2d7, user_id=6844f315435f48f68488a2f0459bf155, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:55.049490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03996a8f-1713-4bb3-ae3e-00754bd20926 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:55.064313 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1051/2129] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:31:55 2022] PUT /identity/v3/projects/4931a52d9cda40b89d15804b69b5a2d7/users/6844f315435f48f68488a2f0459bf155/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:31:55.069512 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f9197-c212-4c04-88f2-f88151f65721 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:55.070002 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f9197-c212-4c04-88f2-f88151f65721 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:55.070082 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f9197-c212-4c04-88f2-f88151f65721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:55.090396 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-259f9197-c212-4c04-88f2-f88151f65721 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:31:55.091679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-259f9197-c212-4c04-88f2-f88151f65721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:55.152954 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1079/2130] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:31:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:31:55.163485 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03996a8f-1713-4bb3-ae3e-00754bd20926 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:31:55.165691 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-03996a8f-1713-4bb3-ae3e-00754bd20926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:31:55.225163 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03996a8f-1713-4bb3-ae3e-00754bd20926 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:31:55.228860 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe369d9-b1e2-43f8-b0d6-2eb32e7759be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:31:55.229414 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe369d9-b1e2-43f8-b0d6-2eb32e7759be service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:31:55.229694 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe369d9-b1e2-43f8-b0d6-2eb32e7759be service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:31:55.229969 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe369d9-b1e2-43f8-b0d6-2eb32e7759be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:31:55.239421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe369d9-b1e2-43f8-b0d6-2eb32e7759be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:31:55.241147 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe369d9-b1e2-43f8-b0d6-2eb32e7759be service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:31:55.251954 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1052/2131] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:31:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:32:00.586892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90b0b2-a502-4451-aed5-e849ad2def10 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:00.587460 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90b0b2-a502-4451-aed5-e849ad2def10 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:00.587460 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90b0b2-a502-4451-aed5-e849ad2def10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:00.639665 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-df90b0b2-a502-4451-aed5-e849ad2def10 None None] MFA Rules not processed for user `6844f315435f48f68488a2f0459bf155`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:00.642019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-df90b0b2-a502-4451-aed5-e849ad2def10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:00.771632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:32:00.788954 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1080/2132] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:00.810228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe369d9-b1e2-43f8-b0d6-2eb32e7759be service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:00.811837 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6fe369d9-b1e2-43f8-b0d6-2eb32e7759be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:00.912125 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:32:00.920564 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe369d9-b1e2-43f8-b0d6-2eb32e7759be service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:00.927550 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d7056-dc3d-4022-9f62-cc7a887166df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:00.928178 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d7056-dc3d-4022-9f62-cc7a887166df service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:00.928498 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d7056-dc3d-4022-9f62-cc7a887166df service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:00.928773 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d7056-dc3d-4022-9f62-cc7a887166df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:00.937238 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d7056-dc3d-4022-9f62-cc7a887166df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:00.939463 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d7056-dc3d-4022-9f62-cc7a887166df service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:00.949477 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1053/2133] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:32:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:32:01.713595 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90b0b2-a502-4451-aed5-e849ad2def10 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:01.715251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-df90b0b2-a502-4451-aed5-e849ad2def10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:01.739677 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90b0b2-a502-4451-aed5-e849ad2def10 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:01.749112 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0578ef1a-3a03-4202-aa3f-e1cd38833cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:01.751440 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578ef1a-3a03-4202-aa3f-e1cd38833cfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:01.752975 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578ef1a-3a03-4202-aa3f-e1cd38833cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:01.753452 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578ef1a-3a03-4202-aa3f-e1cd38833cfd admin admin] PATH_INFO: `/v3/users/576ffe2ec3734f6fa202ef4eca81adad` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:01.755215 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0578ef1a-3a03-4202-aa3f-e1cd38833cfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=576ffe2ec3734f6fa202ef4eca81adad)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:01.758221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0578ef1a-3a03-4202-aa3f-e1cd38833cfd admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:01.869265 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0578ef1a-3a03-4202-aa3f-e1cd38833cfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '576ffe2ec3734f6fa202ef4eca81adad'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:01.893680 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0578ef1a-3a03-4202-aa3f-e1cd38833cfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '576ffe2ec3734f6fa202ef4eca81adad'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:01.899011 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0578ef1a-3a03-4202-aa3f-e1cd38833cfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '576ffe2ec3734f6fa202ef4eca81adad'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:01.901230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0578ef1a-3a03-4202-aa3f-e1cd38833cfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '576ffe2ec3734f6fa202ef4eca81adad'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:01.923917 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0578ef1a-3a03-4202-aa3f-e1cd38833cfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '576ffe2ec3734f6fa202ef4eca81adad'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:01.948827 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1081/2134] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:32:01 2022] DELETE /identity/v3/users/576ffe2ec3734f6fa202ef4eca81adad => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:32:01.953369 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822e034-eb2a-4504-817d-1fd487368e54 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:01.954968 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822e034-eb2a-4504-817d-1fd487368e54 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:01.955068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822e034-eb2a-4504-817d-1fd487368e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:02.002665 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c822e034-eb2a-4504-817d-1fd487368e54 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:02.007348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c822e034-eb2a-4504-817d-1fd487368e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:02.166990 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1054/2135] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:32:02.187207 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0578ef1a-3a03-4202-aa3f-e1cd38833cfd admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:02.190581 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0578ef1a-3a03-4202-aa3f-e1cd38833cfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:02.335611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0578ef1a-3a03-4202-aa3f-e1cd38833cfd admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:02.340591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb42d742-3051-4732-ad9c-5c8984e04d6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:02.341497 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42d742-3051-4732-ad9c-5c8984e04d6f service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:02.341775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42d742-3051-4732-ad9c-5c8984e04d6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:02.342620 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42d742-3051-4732-ad9c-5c8984e04d6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:02.352049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb42d742-3051-4732-ad9c-5c8984e04d6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:02.353668 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb42d742-3051-4732-ad9c-5c8984e04d6f service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:02.365586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1082/2136] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:32:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:32:03.109438 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822e034-eb2a-4504-817d-1fd487368e54 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:03.112024 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c822e034-eb2a-4504-817d-1fd487368e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:03.125544 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822e034-eb2a-4504-817d-1fd487368e54 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:03.130760 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd9883-7344-4aaf-b1f0-9f7564a76d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:03.131589 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd9883-7344-4aaf-b1f0-9f7564a76d5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:03.131867 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd9883-7344-4aaf-b1f0-9f7564a76d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:03.132140 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd9883-7344-4aaf-b1f0-9f7564a76d5d admin admin] PATH_INFO: `/v3/projects/cdd0f4290eaf45c981acb531b818b55a` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:03.133325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd9883-7344-4aaf-b1f0-9f7564a76d5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdd0f4290eaf45c981acb531b818b55a)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:03.135389 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd9883-7344-4aaf-b1f0-9f7564a76d5d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:03.233577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-00bd9883-7344-4aaf-b1f0-9f7564a76d5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdd0f4290eaf45c981acb531b818b55a', 'request_id': 'req-00bd9883-7344-4aaf-b1f0-9f7564a76d5d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:03.244576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-00bd9883-7344-4aaf-b1f0-9f7564a76d5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdd0f4290eaf45c981acb531b818b55a', 'request_id': 'req-00bd9883-7344-4aaf-b1f0-9f7564a76d5d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:03.259371 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-00bd9883-7344-4aaf-b1f0-9f7564a76d5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdd0f4290eaf45c981acb531b818b55a', 'request_id': 'req-00bd9883-7344-4aaf-b1f0-9f7564a76d5d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:03.275451 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-00bd9883-7344-4aaf-b1f0-9f7564a76d5d admin admin] The token cache is being invalidate because project cdd0f4290eaf45c981acb531b818b55a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:32:03.276526 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-00bd9883-7344-4aaf-b1f0-9f7564a76d5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:03.277098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-00bd9883-7344-4aaf-b1f0-9f7564a76d5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:03.278861 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1055/2137] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:32:03 2022] DELETE /identity/v3/projects/cdd0f4290eaf45c981acb531b818b55a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:32:03.284166 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e61243-6c2f-4033-9738-c8e55bbcc80a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:03.285235 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e61243-6c2f-4033-9738-c8e55bbcc80a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:03.285521 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e61243-6c2f-4033-9738-c8e55bbcc80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:03.316572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-66e61243-6c2f-4033-9738-c8e55bbcc80a None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:03.317787 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-66e61243-6c2f-4033-9738-c8e55bbcc80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:03.398605 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1083/2138] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:32:03.414096 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455434d-d548-4e41-b0a4-a74b4608152e None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:03.414600 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455434d-d548-4e41-b0a4-a74b4608152e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:03.415024 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455434d-d548-4e41-b0a4-a74b4608152e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:03.449223 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-7455434d-d548-4e41-b0a4-a74b4608152e None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:03.450679 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7455434d-d548-4e41-b0a4-a74b4608152e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:03.467149 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1056/2139] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:03.467449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e61243-6c2f-4033-9738-c8e55bbcc80a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:03.491438 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e61243-6c2f-4033-9738-c8e55bbcc80a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:03.506951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded2d45-c228-42a3-8065-a7e721eb2495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:03.507803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded2d45-c228-42a3-8065-a7e721eb2495 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:03.508230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded2d45-c228-42a3-8065-a7e721eb2495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:03.508693 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded2d45-c228-42a3-8065-a7e721eb2495 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:03.509542 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded2d45-c228-42a3-8065-a7e721eb2495 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:32:03.510026 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded2d45-c228-42a3-8065-a7e721eb2495 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:03.512157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded2d45-c228-42a3-8065-a7e721eb2495 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:03.543884 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1084/2140] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:32:03.545069 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455434d-d548-4e41-b0a4-a74b4608152e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:03.562327 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455434d-d548-4e41-b0a4-a74b4608152e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:03.567780 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f952c6a-26db-44df-8094-3cb71b4968b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:03.569060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f952c6a-26db-44df-8094-3cb71b4968b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:03.569413 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f952c6a-26db-44df-8094-3cb71b4968b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:03.569757 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f952c6a-26db-44df-8094-3cb71b4968b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:03.570500 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f952c6a-26db-44df-8094-3cb71b4968b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:03.572660 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f952c6a-26db-44df-8094-3cb71b4968b4 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:03.625192 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1057/2141] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:32:03 2022] POST /identity/v3/projects => generated 375 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:32:03.630001 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded2d45-c228-42a3-8065-a7e721eb2495 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:03.638510 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:32:03.647549 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded2d45-c228-42a3-8065-a7e721eb2495 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:03.759111 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262d76a-57c1-4f59-8d09-1b3ed6dbff73 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:03.759484 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262d76a-57c1-4f59-8d09-1b3ed6dbff73 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:03.759833 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262d76a-57c1-4f59-8d09-1b3ed6dbff73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:03.769103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d52ab-bc22-4a51-850c-9b230ec602c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:03.770717 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d52ab-bc22-4a51-850c-9b230ec602c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:03.771195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d52ab-bc22-4a51-850c-9b230ec602c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:03.771895 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d52ab-bc22-4a51-850c-9b230ec602c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:03.772889 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085d52ab-bc22-4a51-850c-9b230ec602c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:03.775203 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085d52ab-bc22-4a51-850c-9b230ec602c8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:03.790300 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-e262d76a-57c1-4f59-8d09-1b3ed6dbff73 None None] MFA Rules not processed for user `6844f315435f48f68488a2f0459bf155`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:03.791990 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e262d76a-57c1-4f59-8d09-1b3ed6dbff73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:03.825315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1085/2142] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:32:03 2022] POST /identity/v3/users => generated 288 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 12:32:03.829791 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d52ab-bc22-4a51-850c-9b230ec602c8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:03.846425 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d52ab-bc22-4a51-850c-9b230ec602c8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:03.848253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:32:03.853164 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15538886-2500-4130-ac5d-448a0672c044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:03.853910 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15538886-2500-4130-ac5d-448a0672c044 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:03.854565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15538886-2500-4130-ac5d-448a0672c044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:03.854977 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15538886-2500-4130-ac5d-448a0672c044 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:03.855823 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15538886-2500-4130-ac5d-448a0672c044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:03.858308 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15538886-2500-4130-ac5d-448a0672c044 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:03.866319 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1058/2143] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:32:03.872351 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1086/2144] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:32:03.877357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e262d76a-57c1-4f59-8d09-1b3ed6dbff73 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:03.889511 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e262d76a-57c1-4f59-8d09-1b3ed6dbff73 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:03.895111 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065bb67-2eb3-4bca-97f4-a27a9a89aa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:03.895733 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065bb67-2eb3-4bca-97f4-a27a9a89aa8f admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:03.896010 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065bb67-2eb3-4bca-97f4-a27a9a89aa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:03.896280 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065bb67-2eb3-4bca-97f4-a27a9a89aa8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:03.897007 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065bb67-2eb3-4bca-97f4-a27a9a89aa8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:03.898601 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065bb67-2eb3-4bca-97f4-a27a9a89aa8f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:03.910147 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1059/2145] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:32:03.914407 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15538886-2500-4130-ac5d-448a0672c044 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:03.929181 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15538886-2500-4130-ac5d-448a0672c044 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:03.934937 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0792fa2-1f85-4e63-ab6f-658293bc1113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:03.935684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0792fa2-1f85-4e63-ab6f-658293bc1113 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:03.936039 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0792fa2-1f85-4e63-ab6f-658293bc1113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:03.939209 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0792fa2-1f85-4e63-ab6f-658293bc1113 admin admin] PATH_INFO: `/v3/projects/2e9b35d0a9eb47689546ce32b5ebdd64/users/fc33fa9f73ec468891a86a3567f55399/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:03.957514 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0792fa2-1f85-4e63-ab6f-658293bc1113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e9b35d0a9eb47689546ce32b5ebdd64, user_id=fc33fa9f73ec468891a86a3567f55399, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:03.960274 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0792fa2-1f85-4e63-ab6f-658293bc1113 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:03.982587 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1087/2146] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:32:03 2022] PUT /identity/v3/projects/2e9b35d0a9eb47689546ce32b5ebdd64/users/fc33fa9f73ec468891a86a3567f55399/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:32:03.988633 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29563861-38f5-49d4-b6cf-9358b2390c8e None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:03.991281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29563861-38f5-49d4-b6cf-9358b2390c8e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:03.991722 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29563861-38f5-49d4-b6cf-9358b2390c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:04.050292 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-29563861-38f5-49d4-b6cf-9358b2390c8e None None] MFA Rules not processed for user `fc33fa9f73ec468891a86a3567f55399`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:04.051528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-29563861-38f5-49d4-b6cf-9358b2390c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:04.134034 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1060/2147] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:04.145821 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0792fa2-1f85-4e63-ab6f-658293bc1113 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:04.147652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a0792fa2-1f85-4e63-ab6f-658293bc1113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:04.231937 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0792fa2-1f85-4e63-ab6f-658293bc1113 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:04.236083 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e5398-4c5c-4815-b2db-0ca500d26254 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:04.237209 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e5398-4c5c-4815-b2db-0ca500d26254 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:04.237495 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e5398-4c5c-4815-b2db-0ca500d26254 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:04.237842 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e5398-4c5c-4815-b2db-0ca500d26254 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:04.254716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e5398-4c5c-4815-b2db-0ca500d26254 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:04.256275 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e5398-4c5c-4815-b2db-0ca500d26254 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:04.278896 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1088/2148] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:32:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:32:07.521078 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29563861-38f5-49d4-b6cf-9358b2390c8e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:07.590318 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29563861-38f5-49d4-b6cf-9358b2390c8e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:07.595904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d91b5-a0a2-4f2e-a61d-f70217ebea78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:07.596662 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d91b5-a0a2-4f2e-a61d-f70217ebea78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:07.596948 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d91b5-a0a2-4f2e-a61d-f70217ebea78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:07.597281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d91b5-a0a2-4f2e-a61d-f70217ebea78 admin admin] PATH_INFO: `/v3/users/fc33fa9f73ec468891a86a3567f55399` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:07.598443 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d91b5-a0a2-4f2e-a61d-f70217ebea78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc33fa9f73ec468891a86a3567f55399)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:07.600419 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d91b5-a0a2-4f2e-a61d-f70217ebea78 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:07.650359 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a34d91b5-a0a2-4f2e-a61d-f70217ebea78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc33fa9f73ec468891a86a3567f55399'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:07.658734 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a34d91b5-a0a2-4f2e-a61d-f70217ebea78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc33fa9f73ec468891a86a3567f55399'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:07.670354 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:32:07.673321 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a34d91b5-a0a2-4f2e-a61d-f70217ebea78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc33fa9f73ec468891a86a3567f55399'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:07.674347 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a34d91b5-a0a2-4f2e-a61d-f70217ebea78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc33fa9f73ec468891a86a3567f55399'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:07.675064 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a34d91b5-a0a2-4f2e-a61d-f70217ebea78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc33fa9f73ec468891a86a3567f55399'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:07.694291 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1061/2149] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:32:07 2022] DELETE /identity/v3/users/fc33fa9f73ec468891a86a3567f55399 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:32:07.697757 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ef9da-a285-4393-80cb-b42ffe127add None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:07.698310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ef9da-a285-4393-80cb-b42ffe127add None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:07.698310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ef9da-a285-4393-80cb-b42ffe127add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:07.720233 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-326ef9da-a285-4393-80cb-b42ffe127add None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:07.721898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-326ef9da-a285-4393-80cb-b42ffe127add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:07.784001 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:32:07.797114 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1089/2150] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:07.803940 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d91b5-a0a2-4f2e-a61d-f70217ebea78 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:07.805314 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a34d91b5-a0a2-4f2e-a61d-f70217ebea78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:07.867812 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d91b5-a0a2-4f2e-a61d-f70217ebea78 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:07.871458 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be89da-351e-4cee-8fc1-2172d916aa3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:07.872068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be89da-351e-4cee-8fc1-2172d916aa3b service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:07.872384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be89da-351e-4cee-8fc1-2172d916aa3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:07.872662 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be89da-351e-4cee-8fc1-2172d916aa3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:07.882093 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be89da-351e-4cee-8fc1-2172d916aa3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:07.883526 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be89da-351e-4cee-8fc1-2172d916aa3b service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:07.893990 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1062/2151] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:32:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:32:08.227851 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ef9da-a285-4393-80cb-b42ffe127add None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:08.229659 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-326ef9da-a285-4393-80cb-b42ffe127add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:08.243074 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ef9da-a285-4393-80cb-b42ffe127add None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:08.248760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50e8a6-6c16-4d29-9092-515971c408fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:08.249556 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50e8a6-6c16-4d29-9092-515971c408fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:08.249996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50e8a6-6c16-4d29-9092-515971c408fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:08.250940 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50e8a6-6c16-4d29-9092-515971c408fd admin admin] PATH_INFO: `/v3/projects/2e9b35d0a9eb47689546ce32b5ebdd64` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:08.252866 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50e8a6-6c16-4d29-9092-515971c408fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e9b35d0a9eb47689546ce32b5ebdd64)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:08.255262 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50e8a6-6c16-4d29-9092-515971c408fd admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:08.326988 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4f50e8a6-6c16-4d29-9092-515971c408fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e9b35d0a9eb47689546ce32b5ebdd64', 'request_id': 'req-4f50e8a6-6c16-4d29-9092-515971c408fd'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:08.334007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4f50e8a6-6c16-4d29-9092-515971c408fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e9b35d0a9eb47689546ce32b5ebdd64', 'request_id': 'req-4f50e8a6-6c16-4d29-9092-515971c408fd'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:08.342731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4f50e8a6-6c16-4d29-9092-515971c408fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e9b35d0a9eb47689546ce32b5ebdd64', 'request_id': 'req-4f50e8a6-6c16-4d29-9092-515971c408fd'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:08.356319 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4f50e8a6-6c16-4d29-9092-515971c408fd admin admin] The token cache is being invalidate because project 2e9b35d0a9eb47689546ce32b5ebdd64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:32:08.356647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4f50e8a6-6c16-4d29-9092-515971c408fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:08.357395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4f50e8a6-6c16-4d29-9092-515971c408fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:08.358927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1090/2152] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:32:08 2022] DELETE /identity/v3/projects/2e9b35d0a9eb47689546ce32b5ebdd64 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:32:08.364650 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e9b29-e2cc-4c20-a7f2-e4193324b24a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:08.365323 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e9b29-e2cc-4c20-a7f2-e4193324b24a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:08.365468 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571e9b29-e2cc-4c20-a7f2-e4193324b24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:08.385319 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-571e9b29-e2cc-4c20-a7f2-e4193324b24a None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:08.386508 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-571e9b29-e2cc-4c20-a7f2-e4193324b24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:08.453260 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1063/2153] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:08.461626 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de49fb5-a1fc-4cf3-9297-feeb14768ac4 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:08.462005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de49fb5-a1fc-4cf3-9297-feeb14768ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:08.462354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de49fb5-a1fc-4cf3-9297-feeb14768ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:08.482570 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1de49fb5-a1fc-4cf3-9297-feeb14768ac4 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:08.483769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1de49fb5-a1fc-4cf3-9297-feeb14768ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:08.492415 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1091/2154] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:08.496788 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571e9b29-e2cc-4c20-a7f2-e4193324b24a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:08.509752 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571e9b29-e2cc-4c20-a7f2-e4193324b24a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:08.514582 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707205a5-c13d-4686-a8cf-f92f667c3696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:08.515199 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707205a5-c13d-4686-a8cf-f92f667c3696 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:08.515614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707205a5-c13d-4686-a8cf-f92f667c3696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:08.516473 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707205a5-c13d-4686-a8cf-f92f667c3696 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:08.517175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707205a5-c13d-4686-a8cf-f92f667c3696 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:32:08.517476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707205a5-c13d-4686-a8cf-f92f667c3696 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:08.519077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707205a5-c13d-4686-a8cf-f92f667c3696 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:08.530880 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1064/2155] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:32:08.535267 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de49fb5-a1fc-4cf3-9297-feeb14768ac4 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:08.546087 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de49fb5-a1fc-4cf3-9297-feeb14768ac4 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:08.551209 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41193874-2570-4958-a8bb-22a479ca36fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:08.552015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41193874-2570-4958-a8bb-22a479ca36fc admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:08.552403 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41193874-2570-4958-a8bb-22a479ca36fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:08.552865 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41193874-2570-4958-a8bb-22a479ca36fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:08.553695 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41193874-2570-4958-a8bb-22a479ca36fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:08.555890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41193874-2570-4958-a8bb-22a479ca36fc admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:08.598008 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1092/2156] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:32:08 2022] POST /identity/v3/projects => generated 367 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:32:08.605563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707205a5-c13d-4686-a8cf-f92f667c3696 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:08.616571 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707205a5-c13d-4686-a8cf-f92f667c3696 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:08.678312 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25d9e9-ea14-455e-8214-261c2e6a856b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:08.678923 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25d9e9-ea14-455e-8214-261c2e6a856b admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:08.679222 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25d9e9-ea14-455e-8214-261c2e6a856b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:08.679479 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25d9e9-ea14-455e-8214-261c2e6a856b admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:08.680150 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd25d9e9-ea14-455e-8214-261c2e6a856b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:08.682098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd25d9e9-ea14-455e-8214-261c2e6a856b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:08.712005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1065/2157] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:32:08 2022] POST /identity/v3/users => generated 284 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:32:08.717486 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41193874-2570-4958-a8bb-22a479ca36fc admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:08.728213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41193874-2570-4958-a8bb-22a479ca36fc admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:08.732904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84143a2-d63c-48f6-9b18-cf0c1c89cee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:08.733510 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84143a2-d63c-48f6-9b18-cf0c1c89cee4 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:08.733788 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84143a2-d63c-48f6-9b18-cf0c1c89cee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:08.734060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84143a2-d63c-48f6-9b18-cf0c1c89cee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:08.734794 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84143a2-d63c-48f6-9b18-cf0c1c89cee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:08.736476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84143a2-d63c-48f6-9b18-cf0c1c89cee4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:08.746770 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1093/2158] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:32:08.751731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25d9e9-ea14-455e-8214-261c2e6a856b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:08.762176 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25d9e9-ea14-455e-8214-261c2e6a856b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:08.767691 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c2b37-bff8-459f-9102-4b18694110ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:08.768331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c2b37-bff8-459f-9102-4b18694110ec admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:08.768623 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c2b37-bff8-459f-9102-4b18694110ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:08.768895 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c2b37-bff8-459f-9102-4b18694110ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:08.769621 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c2b37-bff8-459f-9102-4b18694110ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:08.771214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c2b37-bff8-459f-9102-4b18694110ec admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:08.772743 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:32:08.784273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1066/2159] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:32:08.788532 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84143a2-d63c-48f6-9b18-cf0c1c89cee4 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:08.799505 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84143a2-d63c-48f6-9b18-cf0c1c89cee4 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:08.804509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dba0d9-ed43-4294-805f-9e7ef5c6bbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:08.805208 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dba0d9-ed43-4294-805f-9e7ef5c6bbd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:08.805495 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dba0d9-ed43-4294-805f-9e7ef5c6bbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:08.805772 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dba0d9-ed43-4294-805f-9e7ef5c6bbd3 admin admin] PATH_INFO: `/v3/projects/ece2310c4aca4005820c9cde0f30e626/users/7a4a563f244b4861bc44c2e4b318c598/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:08.825047 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dba0d9-ed43-4294-805f-9e7ef5c6bbd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ece2310c4aca4005820c9cde0f30e626, user_id=7a4a563f244b4861bc44c2e4b318c598, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:08.827859 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dba0d9-ed43-4294-805f-9e7ef5c6bbd3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:08.842242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1094/2160] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:32:08 2022] PUT /identity/v3/projects/ece2310c4aca4005820c9cde0f30e626/users/7a4a563f244b4861bc44c2e4b318c598/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:32:08.853496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247d865-b942-4936-8b02-8a7e8b07240e None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:08.853804 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247d865-b942-4936-8b02-8a7e8b07240e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:08.854056 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247d865-b942-4936-8b02-8a7e8b07240e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:08.887406 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-b247d865-b942-4936-8b02-8a7e8b07240e None None] MFA Rules not processed for user `7a4a563f244b4861bc44c2e4b318c598`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:08.888668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b247d865-b942-4936-8b02-8a7e8b07240e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:08.955197 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1067/2161] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:08.968444 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dba0d9-ed43-4294-805f-9e7ef5c6bbd3 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:08.970197 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-48dba0d9-ed43-4294-805f-9e7ef5c6bbd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:09.061469 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dba0d9-ed43-4294-805f-9e7ef5c6bbd3 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:09.066889 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7f96e-6d78-4b75-8a49-a167875707ad service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7610d6289efb44599d0cee03b93f9938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:09.067517 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7f96e-6d78-4b75-8a49-a167875707ad service glance] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:09.067832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7f96e-6d78-4b75-8a49-a167875707ad service glance] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:09.068108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7f96e-6d78-4b75-8a49-a167875707ad service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:09.076939 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7f96e-6d78-4b75-8a49-a167875707ad service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:09.078389 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7f96e-6d78-4b75-8a49-a167875707ad service glance] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:09.088116 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1095/2162] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:32:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:32:15.430893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d379eb-9e53-4573-9c0e-995357a97cd4 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:15.431469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d379eb-9e53-4573-9c0e-995357a97cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:15.431469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d379eb-9e53-4573-9c0e-995357a97cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:15.453540 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-50d379eb-9e53-4573-9c0e-995357a97cd4 None None] MFA Rules not processed for user `c810b52785b94c0cab4fd9f0d6e65825`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:15.455352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-50d379eb-9e53-4573-9c0e-995357a97cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:15.517563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1068/2163] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Aug 6 12:32:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:15.560208 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7f96e-6d78-4b75-8a49-a167875707ad service glance] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:15.561750 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d6e7f96e-6d78-4b75-8a49-a167875707ad service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:15.567576 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:32:15.636055 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7f96e-6d78-4b75-8a49-a167875707ad service glance] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:15.641020 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e3421-51c2-48c7-bc6d-cb608974e388 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efc792a2655439f8b8070010059b6bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:15.641687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e3421-51c2-48c7-bc6d-cb608974e388 service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:15.641995 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e3421-51c2-48c7-bc6d-cb608974e388 service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:15.642270 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e3421-51c2-48c7-bc6d-cb608974e388 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:15.650387 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e3421-51c2-48c7-bc6d-cb608974e388 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:15.651966 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e3421-51c2-48c7-bc6d-cb608974e388 service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:15.659996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1096/2164] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Aug 6 12:32:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 12:32:16.279766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61e9f3-a6f0-4e55-befb-022859cb26a1 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:16.280048 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61e9f3-a6f0-4e55-befb-022859cb26a1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:16.280340 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61e9f3-a6f0-4e55-befb-022859cb26a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:16.300498 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3b61e9f3-a6f0-4e55-befb-022859cb26a1 None None] MFA Rules not processed for user `c810b52785b94c0cab4fd9f0d6e65825`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:16.301774 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3b61e9f3-a6f0-4e55-befb-022859cb26a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:16.308914 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1069/2165] 10.222.0.20 () {58 vars in 967 bytes} [Sat Aug 6 12:32:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:16.321815 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e3421-51c2-48c7-bc6d-cb608974e388 service swift] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:16.333364 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e3421-51c2-48c7-bc6d-cb608974e388 service swift] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:16.338786 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7e0cc-9190-4a2c-8fc6-6d6a3390cde2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efc792a2655439f8b8070010059b6bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:16.339393 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7e0cc-9190-4a2c-8fc6-6d6a3390cde2 service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:16.339672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7e0cc-9190-4a2c-8fc6-6d6a3390cde2 service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:16.339944 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7e0cc-9190-4a2c-8fc6-6d6a3390cde2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:16.348988 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7e0cc-9190-4a2c-8fc6-6d6a3390cde2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:16.350575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7e0cc-9190-4a2c-8fc6-6d6a3390cde2 service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:16.358562 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1097/2166] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Aug 6 12:32:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 12:32:16.604122 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c47c9-71d5-495c-9a3f-8a7d495b842f None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:16.604663 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c47c9-71d5-495c-9a3f-8a7d495b842f None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:16.604930 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c47c9-71d5-495c-9a3f-8a7d495b842f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:16.627759 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-597c47c9-71d5-495c-9a3f-8a7d495b842f None None] MFA Rules not processed for user `c810b52785b94c0cab4fd9f0d6e65825`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:16.629120 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-597c47c9-71d5-495c-9a3f-8a7d495b842f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:16.636508 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1070/2167] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Aug 6 12:32:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:16.648626 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7e0cc-9190-4a2c-8fc6-6d6a3390cde2 service swift] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:16.659533 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7e0cc-9190-4a2c-8fc6-6d6a3390cde2 service swift] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:16.664555 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575f0e3-bb0b-4fbb-9047-aa0aaa51b0e5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efc792a2655439f8b8070010059b6bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:16.665189 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575f0e3-bb0b-4fbb-9047-aa0aaa51b0e5 service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:16.665469 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575f0e3-bb0b-4fbb-9047-aa0aaa51b0e5 service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:16.666136 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575f0e3-bb0b-4fbb-9047-aa0aaa51b0e5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:16.675132 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575f0e3-bb0b-4fbb-9047-aa0aaa51b0e5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:16.676756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575f0e3-bb0b-4fbb-9047-aa0aaa51b0e5 service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:16.686076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1098/2168] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Aug 6 12:32:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 12:32:17.046897 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d327a-2073-44d9-86b4-63d79d7d606d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:17.047182 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d327a-2073-44d9-86b4-63d79d7d606d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:17.047455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d327a-2073-44d9-86b4-63d79d7d606d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:17.069232 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-cb4d327a-2073-44d9-86b4-63d79d7d606d None None] MFA Rules not processed for user `c810b52785b94c0cab4fd9f0d6e65825`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:17.070566 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-cb4d327a-2073-44d9-86b4-63d79d7d606d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:17.078999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1071/2169] 10.222.0.20 () {58 vars in 967 bytes} [Sat Aug 6 12:32:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:17.101585 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575f0e3-bb0b-4fbb-9047-aa0aaa51b0e5 service swift] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:17.117227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575f0e3-bb0b-4fbb-9047-aa0aaa51b0e5 service swift] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:17.122932 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8aec94-0d41-4301-858c-4914f37c4bd8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efc792a2655439f8b8070010059b6bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:17.123732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8aec94-0d41-4301-858c-4914f37c4bd8 service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:17.124209 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8aec94-0d41-4301-858c-4914f37c4bd8 service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:17.124662 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8aec94-0d41-4301-858c-4914f37c4bd8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:17.134709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8aec94-0d41-4301-858c-4914f37c4bd8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:17.136277 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8aec94-0d41-4301-858c-4914f37c4bd8 service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:17.144927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1099/2170] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Aug 6 12:32:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 12:32:19.405039 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4d327a-2073-44d9-86b4-63d79d7d606d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:19.406536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-cb4d327a-2073-44d9-86b4-63d79d7d606d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:19.483524 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4d327a-2073-44d9-86b4-63d79d7d606d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:19.488276 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d01a6-8aea-4035-83e7-c7efffc7bd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:19.488945 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d01a6-8aea-4035-83e7-c7efffc7bd9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:19.489224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d01a6-8aea-4035-83e7-c7efffc7bd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:19.489498 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d01a6-8aea-4035-83e7-c7efffc7bd9b admin admin] PATH_INFO: `/v3/users/d36ce80ae6a942f1866cd2b0a5fe4c01` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:19.491220 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d01a6-8aea-4035-83e7-c7efffc7bd9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d36ce80ae6a942f1866cd2b0a5fe4c01)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:19.493154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d01a6-8aea-4035-83e7-c7efffc7bd9b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:19.505635 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:32:19.539706 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-2d1d01a6-8aea-4035-83e7-c7efffc7bd9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd36ce80ae6a942f1866cd2b0a5fe4c01'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:19.547128 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-2d1d01a6-8aea-4035-83e7-c7efffc7bd9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd36ce80ae6a942f1866cd2b0a5fe4c01'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:19.559824 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-2d1d01a6-8aea-4035-83e7-c7efffc7bd9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd36ce80ae6a942f1866cd2b0a5fe4c01'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:19.560588 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-2d1d01a6-8aea-4035-83e7-c7efffc7bd9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd36ce80ae6a942f1866cd2b0a5fe4c01'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:19.561189 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-2d1d01a6-8aea-4035-83e7-c7efffc7bd9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd36ce80ae6a942f1866cd2b0a5fe4c01'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:19.573715 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1072/2171] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:32:19 2022] DELETE /identity/v3/users/d36ce80ae6a942f1866cd2b0a5fe4c01 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:32:19.578746 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea04c29-4f9c-43f4-90b8-a0b2d17c4b16 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:19.579281 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea04c29-4f9c-43f4-90b8-a0b2d17c4b16 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:19.579281 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea04c29-4f9c-43f4-90b8-a0b2d17c4b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:19.598691 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-fea04c29-4f9c-43f4-90b8-a0b2d17c4b16 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:19.599864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fea04c29-4f9c-43f4-90b8-a0b2d17c4b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:19.656219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1100/2172] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:32:19.666448 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d01a6-8aea-4035-83e7-c7efffc7bd9b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:19.667872 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2d1d01a6-8aea-4035-83e7-c7efffc7bd9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:19.725923 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d01a6-8aea-4035-83e7-c7efffc7bd9b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:19.729824 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b23e0-9ab0-485e-9e58-91c3d7e9d478 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:19.730439 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b23e0-9ab0-485e-9e58-91c3d7e9d478 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:19.730750 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b23e0-9ab0-485e-9e58-91c3d7e9d478 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:19.731024 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b23e0-9ab0-485e-9e58-91c3d7e9d478 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:19.741355 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4b23e0-9ab0-485e-9e58-91c3d7e9d478 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:19.742713 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4b23e0-9ab0-485e-9e58-91c3d7e9d478 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:19.752904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1073/2173] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:32:19.938115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea04c29-4f9c-43f4-90b8-a0b2d17c4b16 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:19.940061 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fea04c29-4f9c-43f4-90b8-a0b2d17c4b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:19.952434 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea04c29-4f9c-43f4-90b8-a0b2d17c4b16 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:19.958121 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e5af0-ae71-4aa9-b5b6-1820ee36bab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:19.958773 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e5af0-ae71-4aa9-b5b6-1820ee36bab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:19.959047 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e5af0-ae71-4aa9-b5b6-1820ee36bab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:19.959320 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e5af0-ae71-4aa9-b5b6-1820ee36bab6 admin admin] PATH_INFO: `/v3/projects/c1cbbde8a7a342d6a74e14d9010e91e0` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:19.960965 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e5af0-ae71-4aa9-b5b6-1820ee36bab6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1cbbde8a7a342d6a74e14d9010e91e0)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:19.962945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e5af0-ae71-4aa9-b5b6-1820ee36bab6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:20.023652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:32:20.025209 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-fd8e5af0-ae71-4aa9-b5b6-1820ee36bab6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1cbbde8a7a342d6a74e14d9010e91e0', 'request_id': 'req-fd8e5af0-ae71-4aa9-b5b6-1820ee36bab6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:20.031416 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-fd8e5af0-ae71-4aa9-b5b6-1820ee36bab6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1cbbde8a7a342d6a74e14d9010e91e0', 'request_id': 'req-fd8e5af0-ae71-4aa9-b5b6-1820ee36bab6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:20.038469 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-fd8e5af0-ae71-4aa9-b5b6-1820ee36bab6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1cbbde8a7a342d6a74e14d9010e91e0', 'request_id': 'req-fd8e5af0-ae71-4aa9-b5b6-1820ee36bab6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:20.049870 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-fd8e5af0-ae71-4aa9-b5b6-1820ee36bab6 admin admin] The token cache is being invalidate because project c1cbbde8a7a342d6a74e14d9010e91e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:32:20.050159 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-fd8e5af0-ae71-4aa9-b5b6-1820ee36bab6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:20.050786 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-fd8e5af0-ae71-4aa9-b5b6-1820ee36bab6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:20.052289 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1101/2174] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:32:19 2022] DELETE /identity/v3/projects/c1cbbde8a7a342d6a74e14d9010e91e0 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:32:20.057322 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29694f4-736e-4f4d-bc11-37a84f0223c1 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:20.057613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29694f4-736e-4f4d-bc11-37a84f0223c1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:20.057892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29694f4-736e-4f4d-bc11-37a84f0223c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:20.079674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-e29694f4-736e-4f4d-bc11-37a84f0223c1 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:20.080905 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e29694f4-736e-4f4d-bc11-37a84f0223c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:20.150013 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1074/2175] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:20.157996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76b785-0f1d-40c1-9ef3-6ddd8d42a763 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:20.158358 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76b785-0f1d-40c1-9ef3-6ddd8d42a763 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:20.158638 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76b785-0f1d-40c1-9ef3-6ddd8d42a763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:20.178820 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8b76b785-0f1d-40c1-9ef3-6ddd8d42a763 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:20.180090 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8b76b785-0f1d-40c1-9ef3-6ddd8d42a763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:20.188205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1102/2176] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:32:20.193095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29694f4-736e-4f4d-bc11-37a84f0223c1 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:20.202864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29694f4-736e-4f4d-bc11-37a84f0223c1 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:20.207943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c9913-526d-4b8d-b7fb-03c78f50b266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:20.208625 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c9913-526d-4b8d-b7fb-03c78f50b266 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:20.208867 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c9913-526d-4b8d-b7fb-03c78f50b266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:20.209135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c9913-526d-4b8d-b7fb-03c78f50b266 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:20.209825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6c9913-526d-4b8d-b7fb-03c78f50b266 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:32:20.210158 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6c9913-526d-4b8d-b7fb-03c78f50b266 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:20.211771 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6c9913-526d-4b8d-b7fb-03c78f50b266 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:20.223930 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1075/2177] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:32:20.228461 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76b785-0f1d-40c1-9ef3-6ddd8d42a763 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:20.241677 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76b785-0f1d-40c1-9ef3-6ddd8d42a763 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:20.246589 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7bf41-6939-47ef-9b81-b2142636b5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:20.247291 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7bf41-6939-47ef-9b81-b2142636b5c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:20.247640 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7bf41-6939-47ef-9b81-b2142636b5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:20.247983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7bf41-6939-47ef-9b81-b2142636b5c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:20.248769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a7bf41-6939-47ef-9b81-b2142636b5c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:20.250825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a7bf41-6939-47ef-9b81-b2142636b5c5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:20.282220 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1103/2178] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:32:20 2022] POST /identity/v3/projects => generated 399 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:32:20.283922 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c9913-526d-4b8d-b7fb-03c78f50b266 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:20.296501 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c9913-526d-4b8d-b7fb-03c78f50b266 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:20.322750 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:32:20.367676 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c2907-1721-4bd6-adf3-bb64a9a80e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:20.368287 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c2907-1721-4bd6-adf3-bb64a9a80e6b admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:20.368588 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c2907-1721-4bd6-adf3-bb64a9a80e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:20.368863 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c2907-1721-4bd6-adf3-bb64a9a80e6b admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:20.369528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c2907-1721-4bd6-adf3-bb64a9a80e6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:20.371729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c2907-1721-4bd6-adf3-bb64a9a80e6b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:20.397381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1076/2179] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:32:20 2022] POST /identity/v3/users => generated 300 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 12:32:20.401787 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7bf41-6939-47ef-9b81-b2142636b5c5 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:20.415547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7bf41-6939-47ef-9b81-b2142636b5c5 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:20.418223 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72deeb3d-bab7-4881-b165-9ef95f68630b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:20.418902 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72deeb3d-bab7-4881-b165-9ef95f68630b admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:20.419185 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72deeb3d-bab7-4881-b165-9ef95f68630b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:20.419454 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72deeb3d-bab7-4881-b165-9ef95f68630b admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:20.420216 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72deeb3d-bab7-4881-b165-9ef95f68630b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:20.422163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72deeb3d-bab7-4881-b165-9ef95f68630b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:20.432533 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1104/2180] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:32:20.436787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c2907-1721-4bd6-adf3-bb64a9a80e6b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:20.447637 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c2907-1721-4bd6-adf3-bb64a9a80e6b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:20.453677 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b19264-3723-42a0-addc-35f41ed2b564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:20.453944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b19264-3723-42a0-addc-35f41ed2b564 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:20.454746 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b19264-3723-42a0-addc-35f41ed2b564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:20.454746 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b19264-3723-42a0-addc-35f41ed2b564 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:20.455843 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b19264-3723-42a0-addc-35f41ed2b564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:20.457488 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b19264-3723-42a0-addc-35f41ed2b564 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:20.467731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1077/2181] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:32:20.472057 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72deeb3d-bab7-4881-b165-9ef95f68630b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:20.483112 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72deeb3d-bab7-4881-b165-9ef95f68630b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:20.487785 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91f764-2a0f-4c44-bd49-e613ce74ea35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:20.488474 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91f764-2a0f-4c44-bd49-e613ce74ea35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:20.488753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91f764-2a0f-4c44-bd49-e613ce74ea35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:20.489029 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91f764-2a0f-4c44-bd49-e613ce74ea35 admin admin] PATH_INFO: `/v3/projects/b7dc54bf53f145a9a7ea9a06b77f68a1/users/f8100d0e7aaf42e6b6c9b8637a6aed96/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:20.506529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db91f764-2a0f-4c44-bd49-e613ce74ea35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7dc54bf53f145a9a7ea9a06b77f68a1, user_id=f8100d0e7aaf42e6b6c9b8637a6aed96, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:20.510025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db91f764-2a0f-4c44-bd49-e613ce74ea35 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:20.527064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1105/2182] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:32:20 2022] PUT /identity/v3/projects/b7dc54bf53f145a9a7ea9a06b77f68a1/users/f8100d0e7aaf42e6b6c9b8637a6aed96/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:32:20.538427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9f941-fb3c-456d-8077-52fe26adaf0a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:20.538712 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9f941-fb3c-456d-8077-52fe26adaf0a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:20.538986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9f941-fb3c-456d-8077-52fe26adaf0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:20.567169 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-44f9f941-fb3c-456d-8077-52fe26adaf0a None None] MFA Rules not processed for user `f8100d0e7aaf42e6b6c9b8637a6aed96`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:20.568406 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-44f9f941-fb3c-456d-8077-52fe26adaf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:20.633769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1078/2183] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:20 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:20.645520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91f764-2a0f-4c44-bd49-e613ce74ea35 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:20.647220 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-db91f764-2a0f-4c44-bd49-e613ce74ea35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:20.708006 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91f764-2a0f-4c44-bd49-e613ce74ea35 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:20.711843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9abb53-112f-4c22-b734-530700470e6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:20.712564 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9abb53-112f-4c22-b734-530700470e6b service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:20.712912 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9abb53-112f-4c22-b734-530700470e6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:20.713245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9abb53-112f-4c22-b734-530700470e6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:20.723599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9abb53-112f-4c22-b734-530700470e6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:20.725068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9abb53-112f-4c22-b734-530700470e6b service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:20.738446 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1106/2184] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:32:20 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:32:30.630860 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=57,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:32:30.735494 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=60,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:32:35.107030 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9f941-fb3c-456d-8077-52fe26adaf0a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:35.167857 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9f941-fb3c-456d-8077-52fe26adaf0a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:35.172419 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7972638-37c0-468a-a97e-f3458706828b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:35.173060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7972638-37c0-468a-a97e-f3458706828b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:35.173340 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7972638-37c0-468a-a97e-f3458706828b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:35.173616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7972638-37c0-468a-a97e-f3458706828b admin admin] PATH_INFO: `/v3/users/f8100d0e7aaf42e6b6c9b8637a6aed96` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:35.175291 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7972638-37c0-468a-a97e-f3458706828b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8100d0e7aaf42e6b6c9b8637a6aed96)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:35.177253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7972638-37c0-468a-a97e-f3458706828b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:35.221787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c7972638-37c0-468a-a97e-f3458706828b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8100d0e7aaf42e6b6c9b8637a6aed96'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:35.228998 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c7972638-37c0-468a-a97e-f3458706828b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8100d0e7aaf42e6b6c9b8637a6aed96'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:35.241397 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c7972638-37c0-468a-a97e-f3458706828b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8100d0e7aaf42e6b6c9b8637a6aed96'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:35.242057 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c7972638-37c0-468a-a97e-f3458706828b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8100d0e7aaf42e6b6c9b8637a6aed96'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:35.242665 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c7972638-37c0-468a-a97e-f3458706828b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8100d0e7aaf42e6b6c9b8637a6aed96'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:35.256961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1079/2185] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:32:35 2022] DELETE /identity/v3/users/f8100d0e7aaf42e6b6c9b8637a6aed96 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:32:35.261599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5df20d-1ea2-4ac9-988f-0d76387f9dd4 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:35.262098 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5df20d-1ea2-4ac9-988f-0d76387f9dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:35.262248 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5df20d-1ea2-4ac9-988f-0d76387f9dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:35.283847 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ab5df20d-1ea2-4ac9-988f-0d76387f9dd4 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:35.285071 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ab5df20d-1ea2-4ac9-988f-0d76387f9dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:35.345388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1107/2186] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:35.355712 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7972638-37c0-468a-a97e-f3458706828b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:35.357245 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c7972638-37c0-468a-a97e-f3458706828b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:35.417313 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7972638-37c0-468a-a97e-f3458706828b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:35.420886 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e2978-3164-46d1-9644-b04ed8445527 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:35.421506 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e2978-3164-46d1-9644-b04ed8445527 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:35.421787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e2978-3164-46d1-9644-b04ed8445527 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:35.422061 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e2978-3164-46d1-9644-b04ed8445527 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:35.432329 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e2978-3164-46d1-9644-b04ed8445527 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:35.433804 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e2978-3164-46d1-9644-b04ed8445527 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:35.443842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1080/2187] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:32:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:32:35.718544 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5df20d-1ea2-4ac9-988f-0d76387f9dd4 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:35.721906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ab5df20d-1ea2-4ac9-988f-0d76387f9dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:35.745954 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5df20d-1ea2-4ac9-988f-0d76387f9dd4 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:35.752719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f73235-c903-4045-9f7e-24f4fa8073f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:35.753701 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f73235-c903-4045-9f7e-24f4fa8073f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:35.754161 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f73235-c903-4045-9f7e-24f4fa8073f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:35.754612 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f73235-c903-4045-9f7e-24f4fa8073f4 admin admin] PATH_INFO: `/v3/projects/b7dc54bf53f145a9a7ea9a06b77f68a1` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:35.756197 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f73235-c903-4045-9f7e-24f4fa8073f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7dc54bf53f145a9a7ea9a06b77f68a1)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:35.758280 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f73235-c903-4045-9f7e-24f4fa8073f4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:35.839584 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b3f73235-c903-4045-9f7e-24f4fa8073f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7dc54bf53f145a9a7ea9a06b77f68a1', 'request_id': 'req-b3f73235-c903-4045-9f7e-24f4fa8073f4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:35.846647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b3f73235-c903-4045-9f7e-24f4fa8073f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7dc54bf53f145a9a7ea9a06b77f68a1', 'request_id': 'req-b3f73235-c903-4045-9f7e-24f4fa8073f4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:35.855678 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b3f73235-c903-4045-9f7e-24f4fa8073f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7dc54bf53f145a9a7ea9a06b77f68a1', 'request_id': 'req-b3f73235-c903-4045-9f7e-24f4fa8073f4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:35.873880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b3f73235-c903-4045-9f7e-24f4fa8073f4 admin admin] The token cache is being invalidate because project b7dc54bf53f145a9a7ea9a06b77f68a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:32:35.874284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b3f73235-c903-4045-9f7e-24f4fa8073f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:35.875000 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b3f73235-c903-4045-9f7e-24f4fa8073f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:35.877138 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1108/2188] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:32:35 2022] DELETE /identity/v3/projects/b7dc54bf53f145a9a7ea9a06b77f68a1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:32:35.881739 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd56b7-462b-4773-9651-4923bbbdb8d3 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:35.882020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd56b7-462b-4773-9651-4923bbbdb8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:35.882290 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd56b7-462b-4773-9651-4923bbbdb8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:35.908954 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-a8cd56b7-462b-4773-9651-4923bbbdb8d3 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:35.910153 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a8cd56b7-462b-4773-9651-4923bbbdb8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:35.928345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:32:35.978335 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1081/2189] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:35.986402 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9889d2-5cad-478c-9e4d-583ccdf1202c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:35.986689 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9889d2-5cad-478c-9e4d-583ccdf1202c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:35.986979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9889d2-5cad-478c-9e4d-583ccdf1202c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:36.011667 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4b9889d2-5cad-478c-9e4d-583ccdf1202c None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:36.012899 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4b9889d2-5cad-478c-9e4d-583ccdf1202c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:36.022020 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1109/2190] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:36.027539 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd56b7-462b-4773-9651-4923bbbdb8d3 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:36.041233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd56b7-462b-4773-9651-4923bbbdb8d3 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:36.045998 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40474b90-3b44-46f6-8f5f-d3180510a716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:36.046613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40474b90-3b44-46f6-8f5f-d3180510a716 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:36.046892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40474b90-3b44-46f6-8f5f-d3180510a716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:36.047184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40474b90-3b44-46f6-8f5f-d3180510a716 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:36.047856 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40474b90-3b44-46f6-8f5f-d3180510a716 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:32:36.048156 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40474b90-3b44-46f6-8f5f-d3180510a716 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:36.049821 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40474b90-3b44-46f6-8f5f-d3180510a716 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:36.061997 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1082/2191] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:32:36.066904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9889d2-5cad-478c-9e4d-583ccdf1202c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:36.076560 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9889d2-5cad-478c-9e4d-583ccdf1202c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:36.082033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e1f71-3a8c-4754-8940-2acae375e41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:36.082589 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e1f71-3a8c-4754-8940-2acae375e41e admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:36.083226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e1f71-3a8c-4754-8940-2acae375e41e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:36.083226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e1f71-3a8c-4754-8940-2acae375e41e admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:36.083788 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e1f71-3a8c-4754-8940-2acae375e41e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:36.085759 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e1f71-3a8c-4754-8940-2acae375e41e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:36.119508 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1110/2192] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:32:36 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:32:36.123549 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40474b90-3b44-46f6-8f5f-d3180510a716 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:36.133843 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40474b90-3b44-46f6-8f5f-d3180510a716 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:36.198865 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa70e5-beb6-4a03-99f2-5d14355911bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:36.199577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa70e5-beb6-4a03-99f2-5d14355911bb admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:36.199862 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa70e5-beb6-4a03-99f2-5d14355911bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:36.200135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa70e5-beb6-4a03-99f2-5d14355911bb admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:36.200824 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa70e5-beb6-4a03-99f2-5d14355911bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:36.202795 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa70e5-beb6-4a03-99f2-5d14355911bb admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:36.229153 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1083/2193] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:32:36 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:32:36.230734 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e1f71-3a8c-4754-8940-2acae375e41e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:36.242278 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e1f71-3a8c-4754-8940-2acae375e41e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:36.246951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1136ea03-5f4b-4d03-b3e5-69e06403b082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:36.247557 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136ea03-5f4b-4d03-b3e5-69e06403b082 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:36.247859 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136ea03-5f4b-4d03-b3e5-69e06403b082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:36.248134 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136ea03-5f4b-4d03-b3e5-69e06403b082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:36.248863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1136ea03-5f4b-4d03-b3e5-69e06403b082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:36.250751 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1136ea03-5f4b-4d03-b3e5-69e06403b082 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:36.262077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1111/2194] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:32:36.266436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa70e5-beb6-4a03-99f2-5d14355911bb admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:36.276908 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa70e5-beb6-4a03-99f2-5d14355911bb admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:36.282066 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870db8b-9997-4bfa-9769-497fde2ef36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:36.282792 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870db8b-9997-4bfa-9769-497fde2ef36d admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:36.283100 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870db8b-9997-4bfa-9769-497fde2ef36d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:36.283494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870db8b-9997-4bfa-9769-497fde2ef36d admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:36.284239 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870db8b-9997-4bfa-9769-497fde2ef36d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:36.285963 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870db8b-9997-4bfa-9769-497fde2ef36d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:36.298190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1084/2195] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:32:36.302454 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1136ea03-5f4b-4d03-b3e5-69e06403b082 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:36.313011 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1136ea03-5f4b-4d03-b3e5-69e06403b082 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:36.318259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc27eb-5619-4f32-aca0-017cbbd266d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:36.318911 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc27eb-5619-4f32-aca0-017cbbd266d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:36.319188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc27eb-5619-4f32-aca0-017cbbd266d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:36.319660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc27eb-5619-4f32-aca0-017cbbd266d3 admin admin] PATH_INFO: `/v3/projects/c6cb1f1b37a34fedbd0db091e7e830ad/users/439c730ea489445998ec6fa480257b5e/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:36.344411 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc27eb-5619-4f32-aca0-017cbbd266d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6cb1f1b37a34fedbd0db091e7e830ad, user_id=439c730ea489445998ec6fa480257b5e, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:36.347618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc27eb-5619-4f32-aca0-017cbbd266d3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:36.368478 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1112/2196] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:32:36 2022] PUT /identity/v3/projects/c6cb1f1b37a34fedbd0db091e7e830ad/users/439c730ea489445998ec6fa480257b5e/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:32:36.370778 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e99b0b-f377-4554-8335-222e14635a61 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:36.371069 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e99b0b-f377-4554-8335-222e14635a61 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:36.371344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e99b0b-f377-4554-8335-222e14635a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:36.391714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-f8e99b0b-f377-4554-8335-222e14635a61 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:36.393017 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f8e99b0b-f377-4554-8335-222e14635a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:36.456585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1085/2197] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:36.466238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc27eb-5619-4f32-aca0-017cbbd266d3 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:36.468011 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-98bc27eb-5619-4f32-aca0-017cbbd266d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:36.490612 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:32:36.528559 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc27eb-5619-4f32-aca0-017cbbd266d3 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:36.532172 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ffdc8-eed5-42d5-8b2a-a2cce294eff4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:36.532822 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ffdc8-eed5-42d5-8b2a-a2cce294eff4 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:36.533654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ffdc8-eed5-42d5-8b2a-a2cce294eff4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:36.533943 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ffdc8-eed5-42d5-8b2a-a2cce294eff4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:36.542959 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ffdc8-eed5-42d5-8b2a-a2cce294eff4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:36.544417 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ffdc8-eed5-42d5-8b2a-a2cce294eff4 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:36.553678 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1113/2198] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:32:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:32:39.634320 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5c089-fcca-4305-b73f-0d11ff8ac79f None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:39.634921 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5c089-fcca-4305-b73f-0d11ff8ac79f None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:39.635058 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5c089-fcca-4305-b73f-0d11ff8ac79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:39.666212 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-77f5c089-fcca-4305-b73f-0d11ff8ac79f None None] MFA Rules not processed for user `7c520a94e20245ca9cc898dee3ab89bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:39.667422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-77f5c089-fcca-4305-b73f-0d11ff8ac79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:39.702253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:32:39.748928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1086/2199] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:41.253072 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d10b7d-0436-4e4a-bb61-cf5d0b132ef3 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:41.253548 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d10b7d-0436-4e4a-bb61-cf5d0b132ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:41.253839 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d10b7d-0436-4e4a-bb61-cf5d0b132ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:41.289763 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-89d10b7d-0436-4e4a-bb61-cf5d0b132ef3 None None] MFA Rules not processed for user `439c730ea489445998ec6fa480257b5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:41.291457 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-89d10b7d-0436-4e4a-bb61-cf5d0b132ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:41.382836 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1114/2200] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:41.387276 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5c089-fcca-4305-b73f-0d11ff8ac79f None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:41.399809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5c089-fcca-4305-b73f-0d11ff8ac79f None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:41.407251 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867ff28-2075-449a-a2aa-69bf44e78bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:41.408787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867ff28-2075-449a-a2aa-69bf44e78bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:41.409071 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867ff28-2075-449a-a2aa-69bf44e78bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:41.409342 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867ff28-2075-449a-a2aa-69bf44e78bc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:41.410236 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867ff28-2075-449a-a2aa-69bf44e78bc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:41.412515 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867ff28-2075-449a-a2aa-69bf44e78bc4 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:41.450343 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1087/2201] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:32:41 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:32:41.451426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d10b7d-0436-4e4a-bb61-cf5d0b132ef3 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:41.468611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d10b7d-0436-4e4a-bb61-cf5d0b132ef3 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:41.536547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd88ce7-e578-418c-94ac-ba91909a34fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:41.537177 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd88ce7-e578-418c-94ac-ba91909a34fb admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:41.537461 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd88ce7-e578-418c-94ac-ba91909a34fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:41.537735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd88ce7-e578-418c-94ac-ba91909a34fb admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:41.538406 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd88ce7-e578-418c-94ac-ba91909a34fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:41.542919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd88ce7-e578-418c-94ac-ba91909a34fb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:41.571771 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1115/2202] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:32:41 2022] POST /identity/v3/users => generated 295 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:32:41.573159 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867ff28-2075-449a-a2aa-69bf44e78bc4 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:41.595128 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867ff28-2075-449a-a2aa-69bf44e78bc4 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:41.601353 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f59df-1b0b-4033-a351-e8f3d0744f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:41.602223 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f59df-1b0b-4033-a351-e8f3d0744f1f admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:41.602650 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f59df-1b0b-4033-a351-e8f3d0744f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:41.603066 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f59df-1b0b-4033-a351-e8f3d0744f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:41.604015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f59df-1b0b-4033-a351-e8f3d0744f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:41.605822 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f59df-1b0b-4033-a351-e8f3d0744f1f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:41.617221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1088/2203] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:32:41.621358 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd88ce7-e578-418c-94ac-ba91909a34fb admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:41.632567 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd88ce7-e578-418c-94ac-ba91909a34fb admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:41.638408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad23f77-4b46-4b56-86d4-3566b6b8d644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:41.639097 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad23f77-4b46-4b56-86d4-3566b6b8d644 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:41.639384 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad23f77-4b46-4b56-86d4-3566b6b8d644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:41.639668 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad23f77-4b46-4b56-86d4-3566b6b8d644 admin admin] PATH_INFO: `/v3/projects/06bd13a5e8654f769cd62b37a7af9639/users/942dd1a2c99c4530a72768958e4890d3/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:41.655319 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad23f77-4b46-4b56-86d4-3566b6b8d644 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06bd13a5e8654f769cd62b37a7af9639, user_id=942dd1a2c99c4530a72768958e4890d3, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:41.658732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad23f77-4b46-4b56-86d4-3566b6b8d644 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:41.671479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1116/2204] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:32:41 2022] PUT /identity/v3/projects/06bd13a5e8654f769cd62b37a7af9639/users/942dd1a2c99c4530a72768958e4890d3/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:32:41.676118 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f59df-1b0b-4033-a351-e8f3d0744f1f admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:41.685995 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f59df-1b0b-4033-a351-e8f3d0744f1f admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:41.743004 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96ef1b-a12f-491a-8a26-8ca0d3e8f278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:41.743621 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96ef1b-a12f-491a-8a26-8ca0d3e8f278 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:41.743904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96ef1b-a12f-491a-8a26-8ca0d3e8f278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:41.744177 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96ef1b-a12f-491a-8a26-8ca0d3e8f278 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:41.744909 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd96ef1b-a12f-491a-8a26-8ca0d3e8f278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:41.746541 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd96ef1b-a12f-491a-8a26-8ca0d3e8f278 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:41.756883 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1089/2205] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:32:41.761055 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad23f77-4b46-4b56-86d4-3566b6b8d644 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:41.771798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad23f77-4b46-4b56-86d4-3566b6b8d644 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:41.776410 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b91c94-99b6-446a-a344-1a421dd7f6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:41.777087 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b91c94-99b6-446a-a344-1a421dd7f6d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:41.777371 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b91c94-99b6-446a-a344-1a421dd7f6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:41.777648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b91c94-99b6-446a-a344-1a421dd7f6d4 admin admin] PATH_INFO: `/v3/projects/06bd13a5e8654f769cd62b37a7af9639/users/942dd1a2c99c4530a72768958e4890d3/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:41.779758 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b91c94-99b6-446a-a344-1a421dd7f6d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06bd13a5e8654f769cd62b37a7af9639, user_id=942dd1a2c99c4530a72768958e4890d3, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:41.782464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b91c94-99b6-446a-a344-1a421dd7f6d4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:41.793625 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1117/2206] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:32:41 2022] PUT /identity/v3/projects/06bd13a5e8654f769cd62b37a7af9639/users/942dd1a2c99c4530a72768958e4890d3/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:32:42.160916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96ef1b-a12f-491a-8a26-8ca0d3e8f278 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:42.162705 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-bd96ef1b-a12f-491a-8a26-8ca0d3e8f278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:42.197785 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96ef1b-a12f-491a-8a26-8ca0d3e8f278 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:42.309832 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05007af2-65d3-4234-81c5-1e22f6f2532d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:42.310812 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05007af2-65d3-4234-81c5-1e22f6f2532d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:42.311256 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05007af2-65d3-4234-81c5-1e22f6f2532d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:42.311813 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05007af2-65d3-4234-81c5-1e22f6f2532d admin admin] PATH_INFO: `/v3/users/7a31153e522c457dad7154a2965db807` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:42.313475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05007af2-65d3-4234-81c5-1e22f6f2532d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a31153e522c457dad7154a2965db807)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:42.316535 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05007af2-65d3-4234-81c5-1e22f6f2532d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:42.387028 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:32:42.401585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-05007af2-65d3-4234-81c5-1e22f6f2532d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a31153e522c457dad7154a2965db807'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:42.411212 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-05007af2-65d3-4234-81c5-1e22f6f2532d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a31153e522c457dad7154a2965db807'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:42.425540 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-05007af2-65d3-4234-81c5-1e22f6f2532d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a31153e522c457dad7154a2965db807'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:42.426423 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-05007af2-65d3-4234-81c5-1e22f6f2532d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a31153e522c457dad7154a2965db807'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:42.427080 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-05007af2-65d3-4234-81c5-1e22f6f2532d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a31153e522c457dad7154a2965db807'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:42.447542 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1090/2207] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:32:42 2022] DELETE /identity/v3/users/7a31153e522c457dad7154a2965db807 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:32:42.454736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af09e5d-ce23-4113-80ac-0e7242594ca5 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:42.455441 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af09e5d-ce23-4113-80ac-0e7242594ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:42.455523 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af09e5d-ce23-4113-80ac-0e7242594ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:42.475533 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:32:42.506636 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5af09e5d-ce23-4113-80ac-0e7242594ca5 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:42.510748 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5af09e5d-ce23-4113-80ac-0e7242594ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:42.622226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1118/2208] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:42.637774 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05007af2-65d3-4234-81c5-1e22f6f2532d admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:42.640028 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-05007af2-65d3-4234-81c5-1e22f6f2532d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:42.734141 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05007af2-65d3-4234-81c5-1e22f6f2532d admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:42.740610 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f2cef-6326-4d3d-88ea-c73f3eb6be65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:42.741503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f2cef-6326-4d3d-88ea-c73f3eb6be65 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:42.741844 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f2cef-6326-4d3d-88ea-c73f3eb6be65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:42.742262 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f2cef-6326-4d3d-88ea-c73f3eb6be65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:42.756785 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f2cef-6326-4d3d-88ea-c73f3eb6be65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:42.759331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f2cef-6326-4d3d-88ea-c73f3eb6be65 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:42.776581 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1091/2209] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:32:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:32:43.990235 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af09e5d-ce23-4113-80ac-0e7242594ca5 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:43.996866 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5af09e5d-ce23-4113-80ac-0e7242594ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:44.033921 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af09e5d-ce23-4113-80ac-0e7242594ca5 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:44.046369 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd648a1b-c9a2-4f98-a1af-28910b7161c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:44.047728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd648a1b-c9a2-4f98-a1af-28910b7161c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:44.050294 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd648a1b-c9a2-4f98-a1af-28910b7161c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:44.050806 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd648a1b-c9a2-4f98-a1af-28910b7161c4 admin admin] PATH_INFO: `/v3/projects/d63841b5dd4841d99ea3c8f456308095` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:44.054919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd648a1b-c9a2-4f98-a1af-28910b7161c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d63841b5dd4841d99ea3c8f456308095)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:44.070527 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd648a1b-c9a2-4f98-a1af-28910b7161c4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:44.179395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-fd648a1b-c9a2-4f98-a1af-28910b7161c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd63841b5dd4841d99ea3c8f456308095', 'request_id': 'req-fd648a1b-c9a2-4f98-a1af-28910b7161c4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:44.192833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-fd648a1b-c9a2-4f98-a1af-28910b7161c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd63841b5dd4841d99ea3c8f456308095', 'request_id': 'req-fd648a1b-c9a2-4f98-a1af-28910b7161c4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:44.211907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-fd648a1b-c9a2-4f98-a1af-28910b7161c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd63841b5dd4841d99ea3c8f456308095', 'request_id': 'req-fd648a1b-c9a2-4f98-a1af-28910b7161c4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:44.250976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-fd648a1b-c9a2-4f98-a1af-28910b7161c4 admin admin] The token cache is being invalidate because project d63841b5dd4841d99ea3c8f456308095 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:32:44.250976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-fd648a1b-c9a2-4f98-a1af-28910b7161c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:44.253030 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-fd648a1b-c9a2-4f98-a1af-28910b7161c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:44.257326 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1119/2210] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:32:43 2022] DELETE /identity/v3/projects/d63841b5dd4841d99ea3c8f456308095 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:32:47.848509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a8c6b-c9da-4890-ab03-5f497411f1ea None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:47.849470 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a8c6b-c9da-4890-ab03-5f497411f1ea None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:47.850002 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a8c6b-c9da-4890-ab03-5f497411f1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:47.897823 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-408a8c6b-c9da-4890-ab03-5f497411f1ea None None] MFA Rules not processed for user `942dd1a2c99c4530a72768958e4890d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:47.899857 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-408a8c6b-c9da-4890-ab03-5f497411f1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:47.980645 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1092/2211] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:47 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:47.994726 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd648a1b-c9a2-4f98-a1af-28910b7161c4 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:47.996538 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fd648a1b-c9a2-4f98-a1af-28910b7161c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:48.076106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd648a1b-c9a2-4f98-a1af-28910b7161c4 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:48.082407 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52ee50-99ef-4e65-8f3b-0c16afb7f238 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:48.082970 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52ee50-99ef-4e65-8f3b-0c16afb7f238 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:48.083260 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52ee50-99ef-4e65-8f3b-0c16afb7f238 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:48.083565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52ee50-99ef-4e65-8f3b-0c16afb7f238 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:48.085452 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-df52ee50-99ef-4e65-8f3b-0c16afb7f238 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:48.113097 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:32:48.149769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df52ee50-99ef-4e65-8f3b-0c16afb7f238 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:48.151454 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df52ee50-99ef-4e65-8f3b-0c16afb7f238 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:48.161872 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1120/2212] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:32:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:32:56.770581 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a8c6b-c9da-4890-ab03-5f497411f1ea None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:56.772235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-408a8c6b-c9da-4890-ab03-5f497411f1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:56.821317 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:32:56.844108 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a8c6b-c9da-4890-ab03-5f497411f1ea None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:56.849166 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba6c79-9d31-4b0b-ac9d-9e938ee41311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:56.849900 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba6c79-9d31-4b0b-ac9d-9e938ee41311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:56.850221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba6c79-9d31-4b0b-ac9d-9e938ee41311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:56.850495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba6c79-9d31-4b0b-ac9d-9e938ee41311 admin admin] PATH_INFO: `/v3/users/7a4a563f244b4861bc44c2e4b318c598` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:56.851749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba6c79-9d31-4b0b-ac9d-9e938ee41311 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a4a563f244b4861bc44c2e4b318c598)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:56.854013 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba6c79-9d31-4b0b-ac9d-9e938ee41311 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:56.903949 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3cba6c79-9d31-4b0b-ac9d-9e938ee41311 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a4a563f244b4861bc44c2e4b318c598'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:56.912533 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3cba6c79-9d31-4b0b-ac9d-9e938ee41311 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a4a563f244b4861bc44c2e4b318c598'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:56.926020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3cba6c79-9d31-4b0b-ac9d-9e938ee41311 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a4a563f244b4861bc44c2e4b318c598'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:56.926762 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3cba6c79-9d31-4b0b-ac9d-9e938ee41311 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a4a563f244b4861bc44c2e4b318c598'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:56.927389 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3cba6c79-9d31-4b0b-ac9d-9e938ee41311 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a4a563f244b4861bc44c2e4b318c598'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:56.941826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1093/2213] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:32:56 2022] DELETE /identity/v3/users/7a4a563f244b4861bc44c2e4b318c598 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:32:56.946308 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07358f5a-6099-497a-8af7-409285941450 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:56.946762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07358f5a-6099-497a-8af7-409285941450 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:56.946879 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07358f5a-6099-497a-8af7-409285941450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:56.966402 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-07358f5a-6099-497a-8af7-409285941450 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:56.967674 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-07358f5a-6099-497a-8af7-409285941450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:57.031348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1121/2214] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:57.041457 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba6c79-9d31-4b0b-ac9d-9e938ee41311 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:57.042972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3cba6c79-9d31-4b0b-ac9d-9e938ee41311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:57.100880 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba6c79-9d31-4b0b-ac9d-9e938ee41311 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:57.104511 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62271a-6814-44f9-829f-582f26efbb0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:57.105112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62271a-6814-44f9-829f-582f26efbb0d service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:57.105389 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62271a-6814-44f9-829f-582f26efbb0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:57.105657 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62271a-6814-44f9-829f-582f26efbb0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:57.115893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c62271a-6814-44f9-829f-582f26efbb0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:57.117366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c62271a-6814-44f9-829f-582f26efbb0d service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:57.127639 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1094/2215] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:32:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:32:57.396250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07358f5a-6099-497a-8af7-409285941450 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:57.397680 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-07358f5a-6099-497a-8af7-409285941450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:57.410560 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07358f5a-6099-497a-8af7-409285941450 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:57.417123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca9d08-d68a-4f02-9c93-cb9348d823a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:57.417123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca9d08-d68a-4f02-9c93-cb9348d823a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:57.417123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca9d08-d68a-4f02-9c93-cb9348d823a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:57.417123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca9d08-d68a-4f02-9c93-cb9348d823a1 admin admin] PATH_INFO: `/v3/projects/ece2310c4aca4005820c9cde0f30e626` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:57.418490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca9d08-d68a-4f02-9c93-cb9348d823a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ece2310c4aca4005820c9cde0f30e626)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:57.420608 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca9d08-d68a-4f02-9c93-cb9348d823a1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:57.470421 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-42ca9d08-d68a-4f02-9c93-cb9348d823a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ece2310c4aca4005820c9cde0f30e626', 'request_id': 'req-42ca9d08-d68a-4f02-9c93-cb9348d823a1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:57.475747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-42ca9d08-d68a-4f02-9c93-cb9348d823a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ece2310c4aca4005820c9cde0f30e626', 'request_id': 'req-42ca9d08-d68a-4f02-9c93-cb9348d823a1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:57.483072 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-42ca9d08-d68a-4f02-9c93-cb9348d823a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ece2310c4aca4005820c9cde0f30e626', 'request_id': 'req-42ca9d08-d68a-4f02-9c93-cb9348d823a1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:57.494593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-42ca9d08-d68a-4f02-9c93-cb9348d823a1 admin admin] The token cache is being invalidate because project ece2310c4aca4005820c9cde0f30e626 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:32:57.494883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-42ca9d08-d68a-4f02-9c93-cb9348d823a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:57.495521 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-42ca9d08-d68a-4f02-9c93-cb9348d823a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:32:57.497074 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1122/2216] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:32:57 2022] DELETE /identity/v3/projects/ece2310c4aca4005820c9cde0f30e626 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:32:57.502079 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59cc9f-5597-453e-81aa-18bbccc93071 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:57.502365 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59cc9f-5597-453e-81aa-18bbccc93071 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:57.502643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59cc9f-5597-453e-81aa-18bbccc93071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:57.523671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-1d59cc9f-5597-453e-81aa-18bbccc93071 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:57.524866 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-1d59cc9f-5597-453e-81aa-18bbccc93071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:57.584572 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1095/2217] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:57.592910 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532e0c1-59c6-40c7-a014-79d23b0c86b6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:57.593209 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532e0c1-59c6-40c7-a014-79d23b0c86b6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:57.593484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532e0c1-59c6-40c7-a014-79d23b0c86b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:57.615034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d532e0c1-59c6-40c7-a014-79d23b0c86b6 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:57.616224 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d532e0c1-59c6-40c7-a014-79d23b0c86b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:57.624041 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1123/2218] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:57.628421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59cc9f-5597-453e-81aa-18bbccc93071 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:57.638269 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59cc9f-5597-453e-81aa-18bbccc93071 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:57.643378 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4221e-fd17-45af-9e9e-b77de408b019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:57.643990 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4221e-fd17-45af-9e9e-b77de408b019 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:57.644269 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4221e-fd17-45af-9e9e-b77de408b019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:57.644573 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4221e-fd17-45af-9e9e-b77de408b019 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:57.645261 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4221e-fd17-45af-9e9e-b77de408b019 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 12:32:57.645563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4221e-fd17-45af-9e9e-b77de408b019 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:57.647162 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4221e-fd17-45af-9e9e-b77de408b019 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:57.648623 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:32:57.662054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1096/2219] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 12:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 12:32:57.666173 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d532e0c1-59c6-40c7-a014-79d23b0c86b6 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:57.675799 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d532e0c1-59c6-40c7-a014-79d23b0c86b6 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:57.680402 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700874a-4676-46f7-87e5-4d374ec01488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:57.681000 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700874a-4676-46f7-87e5-4d374ec01488 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:57.681294 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700874a-4676-46f7-87e5-4d374ec01488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:57.681542 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700874a-4676-46f7-87e5-4d374ec01488 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:57.682238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6700874a-4676-46f7-87e5-4d374ec01488 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:57.684596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6700874a-4676-46f7-87e5-4d374ec01488 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:57.712175 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1124/2220] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 12:32:57 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:32:57.716253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4221e-fd17-45af-9e9e-b77de408b019 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:57.727586 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4221e-fd17-45af-9e9e-b77de408b019 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:57.785191 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20b432-9910-40c9-b08f-d599fa6e56b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:57.786065 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20b432-9910-40c9-b08f-d599fa6e56b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:57.786065 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20b432-9910-40c9-b08f-d599fa6e56b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:57.786321 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20b432-9910-40c9-b08f-d599fa6e56b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:57.786990 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20b432-9910-40c9-b08f-d599fa6e56b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:57.788989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20b432-9910-40c9-b08f-d599fa6e56b9 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:57.810282 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1097/2221] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 12:32:57 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 12:32:57.815531 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700874a-4676-46f7-87e5-4d374ec01488 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:57.826130 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6700874a-4676-46f7-87e5-4d374ec01488 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:57.830814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c7979-cbe1-42de-a703-3de0f9f89de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:57.831347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c7979-cbe1-42de-a703-3de0f9f89de1 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:57.831630 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c7979-cbe1-42de-a703-3de0f9f89de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:57.831897 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c7979-cbe1-42de-a703-3de0f9f89de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:57.832736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c7979-cbe1-42de-a703-3de0f9f89de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:57.834385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c7979-cbe1-42de-a703-3de0f9f89de1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:57.845538 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1125/2222] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:32:57.849730 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20b432-9910-40c9-b08f-d599fa6e56b9 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:57.861165 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20b432-9910-40c9-b08f-d599fa6e56b9 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:57.866372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf3d38-4ee0-49af-b119-18e28082127d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:57.866995 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf3d38-4ee0-49af-b119-18e28082127d admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:57.867275 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf3d38-4ee0-49af-b119-18e28082127d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:57.867571 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf3d38-4ee0-49af-b119-18e28082127d admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:57.868263 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf3d38-4ee0-49af-b119-18e28082127d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:57.869926 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf3d38-4ee0-49af-b119-18e28082127d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:57.880907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1098/2223] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 12:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 12:32:57.885244 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c7979-cbe1-42de-a703-3de0f9f89de1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:57.895241 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c7979-cbe1-42de-a703-3de0f9f89de1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:57.899747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d54464-5e63-412e-8102-d50c658633cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:57.900513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d54464-5e63-412e-8102-d50c658633cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:57.900790 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d54464-5e63-412e-8102-d50c658633cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:57.901062 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d54464-5e63-412e-8102-d50c658633cb admin admin] PATH_INFO: `/v3/projects/94cc3ab0fa2d47008fc45f600ff1613a/users/3fc91d2e9cd24694a5aa62e9d579c8e0/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:57.904310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:32:57.918372 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d54464-5e63-412e-8102-d50c658633cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=94cc3ab0fa2d47008fc45f600ff1613a, user_id=3fc91d2e9cd24694a5aa62e9d579c8e0, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:57.921046 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d54464-5e63-412e-8102-d50c658633cb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:57.940410 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1126/2224] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 12:32:57 2022] PUT /identity/v3/projects/94cc3ab0fa2d47008fc45f600ff1613a/users/3fc91d2e9cd24694a5aa62e9d579c8e0/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 12:32:57.959993 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065b4db-e139-4539-98e6-5fe2916024f1 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:57.960215 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065b4db-e139-4539-98e6-5fe2916024f1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:57.960606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065b4db-e139-4539-98e6-5fe2916024f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:57.982360 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-b065b4db-e139-4539-98e6-5fe2916024f1 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:57.983768 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b065b4db-e139-4539-98e6-5fe2916024f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:58.060128 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1099/2225] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:58.072383 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d54464-5e63-412e-8102-d50c658633cb admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:58.076113 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f4d54464-5e63-412e-8102-d50c658633cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:58.152780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d54464-5e63-412e-8102-d50c658633cb admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:58.156422 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084b0b6-3a34-4951-8d49-c99c88994341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:58.157581 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084b0b6-3a34-4951-8d49-c99c88994341 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:58.157863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084b0b6-3a34-4951-8d49-c99c88994341 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:58.158135 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084b0b6-3a34-4951-8d49-c99c88994341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:58.167411 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3084b0b6-3a34-4951-8d49-c99c88994341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:58.168882 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3084b0b6-3a34-4951-8d49-c99c88994341 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:58.179030 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1127/2226] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:32:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:32:58.821703 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1b0db-15df-497a-8392-e4a1470a9e34 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:58.822261 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1b0db-15df-497a-8392-e4a1470a9e34 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:58.822410 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1b0db-15df-497a-8392-e4a1470a9e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:58.852797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c5d1b0db-15df-497a-8392-e4a1470a9e34 None None] MFA Rules not processed for user `3fc91d2e9cd24694a5aa62e9d579c8e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:32:58.854132 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c5d1b0db-15df-497a-8392-e4a1470a9e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:58.924832 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1100/2227] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:32:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:32:58.935665 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084b0b6-3a34-4951-8d49-c99c88994341 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:32:58.937057 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3084b0b6-3a34-4951-8d49-c99c88994341 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:32:59.007904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084b0b6-3a34-4951-8d49-c99c88994341 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:32:59.011605 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3276ffa-652b-4a04-93e6-acd4f514a700 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:32:59.012283 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3276ffa-652b-4a04-93e6-acd4f514a700 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:32:59.012592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3276ffa-652b-4a04-93e6-acd4f514a700 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:32:59.012867 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3276ffa-652b-4a04-93e6-acd4f514a700 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:32:59.022202 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3276ffa-652b-4a04-93e6-acd4f514a700 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:32:59.023745 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3276ffa-652b-4a04-93e6-acd4f514a700 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:32:59.033417 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1128/2228] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:32:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:33:01.486303 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5af344-3a1c-4ac8-a00b-dce3999400cc None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:33:01.486826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5af344-3a1c-4ac8-a00b-dce3999400cc None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:33:01.486826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5af344-3a1c-4ac8-a00b-dce3999400cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:33:01.494220 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:33:01.510609 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ec5af344-3a1c-4ac8-a00b-dce3999400cc None None] MFA Rules not processed for user `3fc91d2e9cd24694a5aa62e9d579c8e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:33:01.511878 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ec5af344-3a1c-4ac8-a00b-dce3999400cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:33:01.521438 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1101/2229] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:33:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:33:09.029099 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=58,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:33:11.505666 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:33:15.060732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19aee7-e9cc-402e-ba29-f2a79310dc89 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:33:15.061172 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19aee7-e9cc-402e-ba29-f2a79310dc89 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:33:15.061295 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19aee7-e9cc-402e-ba29-f2a79310dc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:33:15.083402 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-af19aee7-e9cc-402e-ba29-f2a79310dc89 None None] MFA Rules not processed for user `439c730ea489445998ec6fa480257b5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:33:15.084715 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-af19aee7-e9cc-402e-ba29-f2a79310dc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:33:15.149333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1129/2230] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:33:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:33:25.137306 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=25 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:33:32.507830 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5af344-3a1c-4ac8-a00b-dce3999400cc None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:33:32.509291 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ec5af344-3a1c-4ac8-a00b-dce3999400cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:33:32.521709 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5af344-3a1c-4ac8-a00b-dce3999400cc None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:33:32.525209 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace8ee1-e724-49a3-8e6e-792ababce8ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:33:32.525810 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace8ee1-e724-49a3-8e6e-792ababce8ee service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:33:32.526081 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace8ee1-e724-49a3-8e6e-792ababce8ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:33:32.526347 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace8ee1-e724-49a3-8e6e-792ababce8ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:33:32.528046 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6ace8ee1-e724-49a3-8e6e-792ababce8ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:33:32.590580 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace8ee1-e724-49a3-8e6e-792ababce8ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:33:32.592082 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace8ee1-e724-49a3-8e6e-792ababce8ee service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:33:32.604817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1102/2231] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:33:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:33:35.599494 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42444ee6-ec84-4a82-b701-2137d132b6b7 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:33:35.600049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42444ee6-ec84-4a82-b701-2137d132b6b7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:33:35.600258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42444ee6-ec84-4a82-b701-2137d132b6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:33:35.622067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-42444ee6-ec84-4a82-b701-2137d132b6b7 None None] MFA Rules not processed for user `1125a0b93c9d45ccb2d3f28f188dfd04`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:33:35.623591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-42444ee6-ec84-4a82-b701-2137d132b6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:33:35.682879 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1130/2232] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:33:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:33:36.830639 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace8ee1-e724-49a3-8e6e-792ababce8ee service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:33:36.832082 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6ace8ee1-e724-49a3-8e6e-792ababce8ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:33:36.903241 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace8ee1-e724-49a3-8e6e-792ababce8ee service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:33:36.908423 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64282bd-0421-49d2-8a98-51c0e5d36aa8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2244655ba304b9c980b5e68772eefc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:33:36.909041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64282bd-0421-49d2-8a98-51c0e5d36aa8 service placement] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:33:36.909322 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64282bd-0421-49d2-8a98-51c0e5d36aa8 service placement] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:33:36.909593 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64282bd-0421-49d2-8a98-51c0e5d36aa8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:33:36.911327 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c64282bd-0421-49d2-8a98-51c0e5d36aa8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:33:36.922882 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64282bd-0421-49d2-8a98-51c0e5d36aa8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:33:36.924503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64282bd-0421-49d2-8a98-51c0e5d36aa8 service placement] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:33:36.934269 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1103/2233] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:33:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:33:45.676106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=25 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:33:46.929992 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=62 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:33:58.102248 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d9ca8-9c96-4d7b-9747-f060c9ed955a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:33:58.102758 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d9ca8-9c96-4d7b-9747-f060c9ed955a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:33:58.102956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d9ca8-9c96-4d7b-9747-f060c9ed955a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:33:58.122468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-097d9ca8-9c96-4d7b-9747-f060c9ed955a None None] MFA Rules not processed for user `c810b52785b94c0cab4fd9f0d6e65825`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:33:58.123654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-097d9ca8-9c96-4d7b-9747-f060c9ed955a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:33:58.177462 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1131/2234] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Aug 6 12:33:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:33:58.222100 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64282bd-0421-49d2-8a98-51c0e5d36aa8 service placement] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:33:58.223524 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c64282bd-0421-49d2-8a98-51c0e5d36aa8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:33:58.293096 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64282bd-0421-49d2-8a98-51c0e5d36aa8 service placement] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:33:58.298068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e4ee8-4cdc-4fea-8220-d412164f774a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efc792a2655439f8b8070010059b6bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:33:58.298711 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e4ee8-4cdc-4fea-8220-d412164f774a service swift] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:33:58.298989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e4ee8-4cdc-4fea-8220-d412164f774a service swift] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:33:58.299261 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e4ee8-4cdc-4fea-8220-d412164f774a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:33:58.309290 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e4ee8-4cdc-4fea-8220-d412164f774a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:33:58.309967 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e4ee8-4cdc-4fea-8220-d412164f774a service swift] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:33:58.319048 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1104/2235] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Aug 6 12:33:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 12:34:08.170327 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=25 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:34:08.315521 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=32 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:34:15.737884 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41602e-501d-4687-b4ec-1e7deb7729f4 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:34:15.738761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41602e-501d-4687-b4ec-1e7deb7729f4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:34:15.739017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41602e-501d-4687-b4ec-1e7deb7729f4 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:34:15.740565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1132/2236] 10.222.0.20 () {54 vars in 921 bytes} [Sat Aug 6 12:34:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:34:15.747024 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b6461-710f-4911-93e0-c40421c1f1ac None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:34:15.747564 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b6461-710f-4911-93e0-c40421c1f1ac None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:34:15.747564 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b6461-710f-4911-93e0-c40421c1f1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:34:15.768664 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-141b6461-710f-4911-93e0-c40421c1f1ac None None] MFA Rules not processed for user `524b728bf63c4ca88e0f25e531cf5056`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:34:15.769949 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-141b6461-710f-4911-93e0-c40421c1f1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:34:15.838668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1105/2237] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Aug 6 12:34:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:34:15.851721 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41602e-501d-4687-b4ec-1e7deb7729f4 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:34:15.853722 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1d41602e-501d-4687-b4ec-1e7deb7729f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:34:15.915328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41602e-501d-4687-b4ec-1e7deb7729f4 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:34:15.918838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338887fc-07d5-450b-bb01-3963234d2c5c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7610d6289efb44599d0cee03b93f9938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:34:15.919464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338887fc-07d5-450b-bb01-3963234d2c5c service glance] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:34:15.919720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338887fc-07d5-450b-bb01-3963234d2c5c service glance] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:34:15.919991 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338887fc-07d5-450b-bb01-3963234d2c5c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:34:15.930429 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338887fc-07d5-450b-bb01-3963234d2c5c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:34:15.931855 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338887fc-07d5-450b-bb01-3963234d2c5c service glance] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:34:15.942219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1133/2238] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:34:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:34:16.014883 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1903d50-fa44-4cdb-a0c8-86e69d6451d2 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:34:16.015168 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1903d50-fa44-4cdb-a0c8-86e69d6451d2 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:34:16.015439 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1903d50-fa44-4cdb-a0c8-86e69d6451d2 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:34:16.016844 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1106/2239] 10.222.0.20 () {54 vars in 921 bytes} [Sat Aug 6 12:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:34:16.021750 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7119c-8784-457d-a244-2cc67cf203be None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:34:16.022040 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7119c-8784-457d-a244-2cc67cf203be None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:34:16.022314 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7119c-8784-457d-a244-2cc67cf203be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:34:16.043248 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a5c7119c-8784-457d-a244-2cc67cf203be None None] MFA Rules not processed for user `55563aaa662246c5aaf107145475bd76`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:34:16.044468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a5c7119c-8784-457d-a244-2cc67cf203be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:34:16.052907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1134/2240] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Aug 6 12:34:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:34:16.065026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1903d50-fa44-4cdb-a0c8-86e69d6451d2 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:34:16.076348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1903d50-fa44-4cdb-a0c8-86e69d6451d2 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:34:16.083582 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201ce61-bbd8-4a80-80af-efcc727e5711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:34:16.084281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201ce61-bbd8-4a80-80af-efcc727e5711 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:34:16.084650 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201ce61-bbd8-4a80-80af-efcc727e5711 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:34:16.084985 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201ce61-bbd8-4a80-80af-efcc727e5711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:34:16.092233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201ce61-bbd8-4a80-80af-efcc727e5711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:34:16.093672 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201ce61-bbd8-4a80-80af-efcc727e5711 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:34:16.101970 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1107/2241] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:34:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:34:16.186195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d77f11-a02a-4b50-bdf7-47eb14d7383c None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:34:16.186483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d77f11-a02a-4b50-bdf7-47eb14d7383c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:34:16.186768 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d77f11-a02a-4b50-bdf7-47eb14d7383c None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:34:16.188136 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1135/2242] 10.222.0.20 () {54 vars in 921 bytes} [Sat Aug 6 12:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 12:34:16.194679 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f099a-7cee-4999-8b57-f354c680c1e2 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:34:16.195037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f099a-7cee-4999-8b57-f354c680c1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:34:16.195381 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f099a-7cee-4999-8b57-f354c680c1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:34:16.215980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-584f099a-7cee-4999-8b57-f354c680c1e2 None None] MFA Rules not processed for user `524b728bf63c4ca88e0f25e531cf5056`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:34:16.217601 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-584f099a-7cee-4999-8b57-f354c680c1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:34:16.228045 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1108/2243] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Aug 6 12:34:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:34:16.240426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d77f11-a02a-4b50-bdf7-47eb14d7383c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:34:16.251736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d77f11-a02a-4b50-bdf7-47eb14d7383c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:34:16.255895 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65368d-33f3-432b-92a2-87a05b7e5c04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:34:16.256578 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65368d-33f3-432b-92a2-87a05b7e5c04 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:34:16.256857 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65368d-33f3-432b-92a2-87a05b7e5c04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:34:16.257128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65368d-33f3-432b-92a2-87a05b7e5c04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:34:16.267082 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a65368d-33f3-432b-92a2-87a05b7e5c04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:34:16.268561 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a65368d-33f3-432b-92a2-87a05b7e5c04 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:34:16.279054 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1136/2244] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:34:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:34:20.658371 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca355bf8-9e63-4bcf-b6d5-16a15a7d2f19 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:34:20.659378 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca355bf8-9e63-4bcf-b6d5-16a15a7d2f19 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:34:20.659542 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca355bf8-9e63-4bcf-b6d5-16a15a7d2f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:34:20.678127 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ca355bf8-9e63-4bcf-b6d5-16a15a7d2f19 None None] MFA Rules not processed for user `c810b52785b94c0cab4fd9f0d6e65825`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:34:20.679345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ca355bf8-9e63-4bcf-b6d5-16a15a7d2f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:34:20.688064 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1109/2245] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Aug 6 12:34:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:34:20.708806 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65368d-33f3-432b-92a2-87a05b7e5c04 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:34:20.724271 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65368d-33f3-432b-92a2-87a05b7e5c04 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:34:20.731347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7179b5ff-e9cc-4e70-8ea2-591618d56d7a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efc792a2655439f8b8070010059b6bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:34:20.732436 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179b5ff-e9cc-4e70-8ea2-591618d56d7a service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:34:20.733036 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179b5ff-e9cc-4e70-8ea2-591618d56d7a service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:34:20.733640 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179b5ff-e9cc-4e70-8ea2-591618d56d7a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:34:20.745810 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7179b5ff-e9cc-4e70-8ea2-591618d56d7a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:34:20.747417 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7179b5ff-e9cc-4e70-8ea2-591618d56d7a service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:34:20.756694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1137/2246] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Aug 6 12:34:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 06 12:34:30.672992 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=43 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:34:30.753480 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=41 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:34:50.523513 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318f485-f47b-4092-a869-6ee4df8facce None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:34:50.524045 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318f485-f47b-4092-a869-6ee4df8facce None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:34:50.524045 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318f485-f47b-4092-a869-6ee4df8facce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:34:50.542386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3318f485-f47b-4092-a869-6ee4df8facce None None] MFA Rules not processed for user `3fc91d2e9cd24694a5aa62e9d579c8e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:34:50.543528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3318f485-f47b-4092-a869-6ee4df8facce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:34:50.550922 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1110/2247] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:34:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:34:55.649672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5252127-a2fa-4752-a1e3-457b0caa64f0 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:34:55.650235 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5252127-a2fa-4752-a1e3-457b0caa64f0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:34:55.650235 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5252127-a2fa-4752-a1e3-457b0caa64f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:34:55.670962 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f5252127-a2fa-4752-a1e3-457b0caa64f0 None None] MFA Rules not processed for user `c810b52785b94c0cab4fd9f0d6e65825`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:34:55.672168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f5252127-a2fa-4752-a1e3-457b0caa64f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:34:55.680325 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1138/2248] 10.222.0.20 () {58 vars in 967 bytes} [Sat Aug 6 12:34:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:34:55.693530 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318f485-f47b-4092-a869-6ee4df8facce None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:34:55.705631 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318f485-f47b-4092-a869-6ee4df8facce None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:34:55.710511 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c3e7c-f4bd-4333-9c64-82a1c06e509d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efc792a2655439f8b8070010059b6bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:34:55.711122 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c3e7c-f4bd-4333-9c64-82a1c06e509d service swift] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:34:55.711396 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c3e7c-f4bd-4333-9c64-82a1c06e509d service swift] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:34:55.711666 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c3e7c-f4bd-4333-9c64-82a1c06e509d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:34:55.719782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5c3e7c-f4bd-4333-9c64-82a1c06e509d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:34:55.721925 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5c3e7c-f4bd-4333-9c64-82a1c06e509d service swift] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:34:55.730587 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1111/2249] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Aug 6 12:34:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 12:35:05.666131 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:35:05.727188 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=9 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:35:05.832100 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5252127-a2fa-4752-a1e3-457b0caa64f0 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:35:05.841507 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5252127-a2fa-4752-a1e3-457b0caa64f0 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:35:05.845522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84196a4-954a-414c-8169-555ba4e62fb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:35:05.846896 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84196a4-954a-414c-8169-555ba4e62fb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:35:05.847241 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84196a4-954a-414c-8169-555ba4e62fb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:35:05.847606 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84196a4-954a-414c-8169-555ba4e62fb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:35:05.849477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b84196a4-954a-414c-8169-555ba4e62fb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:35:05.863813 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84196a4-954a-414c-8169-555ba4e62fb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:35:05.865254 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84196a4-954a-414c-8169-555ba4e62fb0 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:35:05.876933 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1139/2250] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:35:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:35:06.199362 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c3e7c-f4bd-4333-9c64-82a1c06e509d service swift] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:35:06.210836 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c3e7c-f4bd-4333-9c64-82a1c06e509d service swift] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:35:06.214459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916de66b-9112-454a-8910-6320a243ce8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:35:06.215065 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916de66b-9112-454a-8910-6320a243ce8f service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:35:06.215460 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916de66b-9112-454a-8910-6320a243ce8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:35:06.215744 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916de66b-9112-454a-8910-6320a243ce8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:35:06.218098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-916de66b-9112-454a-8910-6320a243ce8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:35:06.233017 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916de66b-9112-454a-8910-6320a243ce8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:35:06.234494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916de66b-9112-454a-8910-6320a243ce8f service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:35:06.247506 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1112/2251] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:35:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:35:15.871911 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=6 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:35:16.242291 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=6 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:35:20.040550 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84196a4-954a-414c-8169-555ba4e62fb0 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:35:20.050129 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84196a4-954a-414c-8169-555ba4e62fb0 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:35:20.053636 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d051d9-92c6-4784-999e-0657b7842d52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:35:20.054228 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d051d9-92c6-4784-999e-0657b7842d52 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:35:20.054501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d051d9-92c6-4784-999e-0657b7842d52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:35:20.054771 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d051d9-92c6-4784-999e-0657b7842d52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:35:20.056486 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d0d051d9-92c6-4784-999e-0657b7842d52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:35:20.113897 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d051d9-92c6-4784-999e-0657b7842d52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:35:20.115227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d051d9-92c6-4784-999e-0657b7842d52 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:35:20.124067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1140/2252] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:35:20 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:35:30.119968 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=28 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:35:50.554473 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916de66b-9112-454a-8910-6320a243ce8f service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:35:50.556086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-916de66b-9112-454a-8910-6320a243ce8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:35:50.570948 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916de66b-9112-454a-8910-6320a243ce8f service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:35:50.576366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bcf06-4d37-43f9-b06b-869789d8e505 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:35:50.577027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bcf06-4d37-43f9-b06b-869789d8e505 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:35:50.577305 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bcf06-4d37-43f9-b06b-869789d8e505 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:35:50.577580 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bcf06-4d37-43f9-b06b-869789d8e505 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:35:50.579284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-476bcf06-4d37-43f9-b06b-869789d8e505 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:35:50.589338 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bcf06-4d37-43f9-b06b-869789d8e505 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:35:50.590903 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bcf06-4d37-43f9-b06b-869789d8e505 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:35:50.599686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1113/2253] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:35:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:36:00.595596 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=6 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:36:18.000670 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d051d9-92c6-4784-999e-0657b7842d52 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:36:18.002032 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d0d051d9-92c6-4784-999e-0657b7842d52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:36:18.017330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d051d9-92c6-4784-999e-0657b7842d52 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:36:18.023170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68002193-ad04-4c0a-a21e-89eb362fa840 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:36:18.023766 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68002193-ad04-4c0a-a21e-89eb362fa840 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:36:18.024050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68002193-ad04-4c0a-a21e-89eb362fa840 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:36:18.024343 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68002193-ad04-4c0a-a21e-89eb362fa840 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:36:18.026020 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-68002193-ad04-4c0a-a21e-89eb362fa840 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:36:18.082426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68002193-ad04-4c0a-a21e-89eb362fa840 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:36:18.084578 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68002193-ad04-4c0a-a21e-89eb362fa840 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:36:18.093732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1141/2254] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:36:17 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:36:20.684400 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bcf06-4d37-43f9-b06b-869789d8e505 service nova] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:36:20.694375 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bcf06-4d37-43f9-b06b-869789d8e505 service nova] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:36:20.697965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b11b7-dad3-4be9-852a-bf4ce3524fbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:36:20.698574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b11b7-dad3-4be9-852a-bf4ce3524fbe service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:36:20.698854 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b11b7-dad3-4be9-852a-bf4ce3524fbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:36:20.699127 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b11b7-dad3-4be9-852a-bf4ce3524fbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:36:20.700909 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5a2b11b7-dad3-4be9-852a-bf4ce3524fbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:36:20.710830 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2b11b7-dad3-4be9-852a-bf4ce3524fbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:36:20.712243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2b11b7-dad3-4be9-852a-bf4ce3524fbe service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:36:20.720013 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1114/2255] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:36:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:36:23.233623 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68002193-ad04-4c0a-a21e-89eb362fa840 service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:36:23.251974 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68002193-ad04-4c0a-a21e-89eb362fa840 service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:36:23.259306 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135c0f3-6d5e-466e-8445-7e05aee18f6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:36:23.260006 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135c0f3-6d5e-466e-8445-7e05aee18f6c service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:36:23.260281 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135c0f3-6d5e-466e-8445-7e05aee18f6c service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:36:23.260796 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135c0f3-6d5e-466e-8445-7e05aee18f6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:36:23.264088 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e135c0f3-6d5e-466e-8445-7e05aee18f6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:36:23.278996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135c0f3-6d5e-466e-8445-7e05aee18f6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:36:23.281304 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135c0f3-6d5e-466e-8445-7e05aee18f6c service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:36:23.296256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1142/2256] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:36:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:36:30.716572 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=6 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:36:33.288906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=34 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:36:52.376559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d97439-d99a-42aa-85d8-5bc27447d440 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:36:52.377067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d97439-d99a-42aa-85d8-5bc27447d440 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:36:52.377067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d97439-d99a-42aa-85d8-5bc27447d440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:36:52.396989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-44d97439-d99a-42aa-85d8-5bc27447d440 None None] MFA Rules not processed for user `7c520a94e20245ca9cc898dee3ab89bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:36:52.398863 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-44d97439-d99a-42aa-85d8-5bc27447d440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:36:52.407084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1115/2257] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:36:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:37:02.390737 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:37:48.757387 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135c0f3-6d5e-466e-8445-7e05aee18f6c service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:37:48.769091 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135c0f3-6d5e-466e-8445-7e05aee18f6c service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:37:48.774534 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a134c-c5d6-419c-b24e-f384873314fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:37:48.775132 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a134c-c5d6-419c-b24e-f384873314fb service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:37:48.775447 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a134c-c5d6-419c-b24e-f384873314fb service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:37:48.775791 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a134c-c5d6-419c-b24e-f384873314fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:37:48.777491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d72a134c-c5d6-419c-b24e-f384873314fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:37:48.787669 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a134c-c5d6-419c-b24e-f384873314fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:37:48.789258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a134c-c5d6-419c-b24e-f384873314fb service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:37:48.797921 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1143/2258] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:37:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:37:58.794232 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=6 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:38:22.116458 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d97439-d99a-42aa-85d8-5bc27447d440 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:38:22.117887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-44d97439-d99a-42aa-85d8-5bc27447d440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:38:22.158413 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d97439-d99a-42aa-85d8-5bc27447d440 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:38:22.162536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea47a17-4d42-4759-9eb7-ddd1e0761802 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:38:22.163141 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea47a17-4d42-4759-9eb7-ddd1e0761802 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:38:22.163416 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea47a17-4d42-4759-9eb7-ddd1e0761802 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:38:22.163690 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea47a17-4d42-4759-9eb7-ddd1e0761802 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:38:22.184931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea47a17-4d42-4759-9eb7-ddd1e0761802 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:38:22.186360 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea47a17-4d42-4759-9eb7-ddd1e0761802 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:38:22.194899 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1116/2259] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:38:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:38:22.220729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a134c-c5d6-419c-b24e-f384873314fb service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:38:22.240580 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a134c-c5d6-419c-b24e-f384873314fb service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:38:22.244214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8eb49-facf-49f0-959f-80fcb6e30e17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:38:22.244855 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8eb49-facf-49f0-959f-80fcb6e30e17 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:38:22.245132 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8eb49-facf-49f0-959f-80fcb6e30e17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:38:22.245434 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8eb49-facf-49f0-959f-80fcb6e30e17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:38:22.247192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1ef8eb49-facf-49f0-959f-80fcb6e30e17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:38:22.303923 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8eb49-facf-49f0-959f-80fcb6e30e17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:38:22.305389 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8eb49-facf-49f0-959f-80fcb6e30e17 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:38:22.314128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1144/2260] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:38:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:38:32.190978 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=25 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:38:32.310093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=33 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:39:30.194015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea47a17-4d42-4759-9eb7-ddd1e0761802 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:39:30.195470 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fea47a17-4d42-4759-9eb7-ddd1e0761802 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:39:30.239026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea47a17-4d42-4759-9eb7-ddd1e0761802 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:39:30.244020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ddb36-791f-4fa7-aff5-a102d65083d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2244655ba304b9c980b5e68772eefc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:39:30.244652 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ddb36-791f-4fa7-aff5-a102d65083d8 service placement] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:39:30.244932 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ddb36-791f-4fa7-aff5-a102d65083d8 service placement] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:39:30.245205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ddb36-791f-4fa7-aff5-a102d65083d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:39:30.253448 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ddb36-791f-4fa7-aff5-a102d65083d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:39:30.255056 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ddb36-791f-4fa7-aff5-a102d65083d8 service placement] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:39:30.264635 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1117/2261] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:39:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:39:40.260711 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=18 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:40:05.846787 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8eb49-facf-49f0-959f-80fcb6e30e17 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:40:05.865595 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8eb49-facf-49f0-959f-80fcb6e30e17 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:40:05.869336 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0641db5-5558-4913-86c9-c15d81b54655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:40:05.869950 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0641db5-5558-4913-86c9-c15d81b54655 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:40:05.870234 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0641db5-5558-4913-86c9-c15d81b54655 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:40:05.870511 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0641db5-5558-4913-86c9-c15d81b54655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:40:05.887250 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0641db5-5558-4913-86c9-c15d81b54655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:40:05.888709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0641db5-5558-4913-86c9-c15d81b54655 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:40:05.910290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1145/2262] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:40:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:40:07.014610 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ddb36-791f-4fa7-aff5-a102d65083d8 service placement] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:40:07.023730 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ddb36-791f-4fa7-aff5-a102d65083d8 service placement] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:40:07.027281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b467a2-ecd6-44d6-9e59-e825138d5c48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:40:07.027866 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b467a2-ecd6-44d6-9e59-e825138d5c48 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:40:07.028140 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b467a2-ecd6-44d6-9e59-e825138d5c48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:40:07.028442 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b467a2-ecd6-44d6-9e59-e825138d5c48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:40:07.038824 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b467a2-ecd6-44d6-9e59-e825138d5c48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:40:07.040231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b467a2-ecd6-44d6-9e59-e825138d5c48 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:40:07.051558 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1118/2263] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:40:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:40:15.907695 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=17 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:40:17.046034 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=6 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:40:22.593925 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0641db5-5558-4913-86c9-c15d81b54655 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:40:22.613507 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0641db5-5558-4913-86c9-c15d81b54655 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:40:22.617183 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50aa02-1bb5-43ed-8058-340542ed13a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:40:22.617784 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50aa02-1bb5-43ed-8058-340542ed13a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:40:22.618060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50aa02-1bb5-43ed-8058-340542ed13a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:40:22.618331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50aa02-1bb5-43ed-8058-340542ed13a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:40:22.627380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50aa02-1bb5-43ed-8058-340542ed13a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:40:22.628824 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50aa02-1bb5-43ed-8058-340542ed13a4 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:40:22.656359 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1146/2264] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:40:22 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:40:32.653799 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=17 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:40:51.095164 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b467a2-ecd6-44d6-9e59-e825138d5c48 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:40:51.107050 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b467a2-ecd6-44d6-9e59-e825138d5c48 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:40:51.113211 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39363a-3bec-47fd-9870-479f408e2d8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:40:51.113830 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39363a-3bec-47fd-9870-479f408e2d8e service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:40:51.114107 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39363a-3bec-47fd-9870-479f408e2d8e service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:40:51.114404 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc39363a-3bec-47fd-9870-479f408e2d8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:40:51.122580 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc39363a-3bec-47fd-9870-479f408e2d8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:40:51.124148 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc39363a-3bec-47fd-9870-479f408e2d8e service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:40:51.149916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1119/2265] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:40:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:41:01.147386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=13 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:41:18.622147 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50aa02-1bb5-43ed-8058-340542ed13a4 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:41:18.634767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50aa02-1bb5-43ed-8058-340542ed13a4 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:41:18.640206 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d89b37-8421-4404-90b1-9ebf096a5bde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:41:18.641427 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d89b37-8421-4404-90b1-9ebf096a5bde service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:41:18.641710 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d89b37-8421-4404-90b1-9ebf096a5bde service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:41:18.641985 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d89b37-8421-4404-90b1-9ebf096a5bde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:41:18.650356 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d89b37-8421-4404-90b1-9ebf096a5bde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:41:18.651902 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d89b37-8421-4404-90b1-9ebf096a5bde service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:41:18.677987 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1147/2266] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:41:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:41:21.577860 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39363a-3bec-47fd-9870-479f408e2d8e service nova] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:41:21.587219 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc39363a-3bec-47fd-9870-479f408e2d8e service nova] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:41:21.590721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5870f22-ca6b-4855-bcc7-886bada277d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:41:21.591396 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5870f22-ca6b-4855-bcc7-886bada277d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:41:21.591671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5870f22-ca6b-4855-bcc7-886bada277d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:41:21.591936 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5870f22-ca6b-4855-bcc7-886bada277d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:41:21.599376 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5870f22-ca6b-4855-bcc7-886bada277d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:41:21.600855 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5870f22-ca6b-4855-bcc7-886bada277d1 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:41:21.608767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1120/2267] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:41:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:41:28.675494 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=13 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:41:31.605147 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=6 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:44:23.080514 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d89b37-8421-4404-90b1-9ebf096a5bde service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:44:23.134397 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d89b37-8421-4404-90b1-9ebf096a5bde service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:44:23.138041 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12a2de-423d-4df2-95ec-43b5cd7bd291 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:44:23.138584 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12a2de-423d-4df2-95ec-43b5cd7bd291 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:44:23.138860 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12a2de-423d-4df2-95ec-43b5cd7bd291 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:44:23.139130 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12a2de-423d-4df2-95ec-43b5cd7bd291 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:44:23.141115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ed12a2de-423d-4df2-95ec-43b5cd7bd291 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:44:23.217732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed12a2de-423d-4df2-95ec-43b5cd7bd291 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:44:23.219159 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed12a2de-423d-4df2-95ec-43b5cd7bd291 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:44:23.237458 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1148/2268] 10.222.0.20 () {60 vars in 1396 bytes} [Sat Aug 6 12:44:23 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:44:23.319653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5870f22-ca6b-4855-bcc7-886bada277d1 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:44:23.328100 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5870f22-ca6b-4855-bcc7-886bada277d1 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:44:23.331700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a35b3a-e395-4f86-81d9-6ebd8153ea3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:44:23.332323 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a35b3a-e395-4f86-81d9-6ebd8153ea3d service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:44:23.332600 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a35b3a-e395-4f86-81d9-6ebd8153ea3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:44:23.332922 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a35b3a-e395-4f86-81d9-6ebd8153ea3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:44:23.344104 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a35b3a-e395-4f86-81d9-6ebd8153ea3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:44:23.345573 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a35b3a-e395-4f86-81d9-6ebd8153ea3d service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:44:23.381084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1121/2269] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:44:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:44:23.417301 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12a2de-423d-4df2-95ec-43b5cd7bd291 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:44:23.494976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12a2de-423d-4df2-95ec-43b5cd7bd291 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:44:23.500703 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ffd9c-1cdc-4de8-8375-3663e9648c5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:44:23.501445 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ffd9c-1cdc-4de8-8375-3663e9648c5e service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:44:23.501809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ffd9c-1cdc-4de8-8375-3663e9648c5e service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:44:23.502094 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ffd9c-1cdc-4de8-8375-3663e9648c5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:44:23.525182 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:44:23.571131 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ffd9c-1cdc-4de8-8375-3663e9648c5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:44:23.572762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ffd9c-1cdc-4de8-8375-3663e9648c5e service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:44:23.591363 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1149/2270] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:44:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:44:33.378354 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=13 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:44:33.588703 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=25 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:44:36.155722 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a35b3a-e395-4f86-81d9-6ebd8153ea3d service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:44:36.157297 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f3a35b3a-e395-4f86-81d9-6ebd8153ea3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:44:36.223511 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a35b3a-e395-4f86-81d9-6ebd8153ea3d service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:44:36.228490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73792ef-b369-43f3-8302-1a318b42f2b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2244655ba304b9c980b5e68772eefc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:44:36.229097 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73792ef-b369-43f3-8302-1a318b42f2b6 service placement] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:44:36.229377 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73792ef-b369-43f3-8302-1a318b42f2b6 service placement] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:44:36.229650 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73792ef-b369-43f3-8302-1a318b42f2b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:44:36.231469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b73792ef-b369-43f3-8302-1a318b42f2b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:44:36.243205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73792ef-b369-43f3-8302-1a318b42f2b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:44:36.244813 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73792ef-b369-43f3-8302-1a318b42f2b6 service placement] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:44:36.254401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1122/2271] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:44:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:44:46.250360 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=32 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:46:23.505537 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ffd9c-1cdc-4de8-8375-3663e9648c5e service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:46:23.514684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ffd9c-1cdc-4de8-8375-3663e9648c5e service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:46:23.518613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea63b3c-d663-451d-86d7-8dee368c1b5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:46:23.519697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea63b3c-d663-451d-86d7-8dee368c1b5b service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:46:23.519971 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea63b3c-d663-451d-86d7-8dee368c1b5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:46:23.520239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea63b3c-d663-451d-86d7-8dee368c1b5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:46:23.522067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7ea63b3c-d663-451d-86d7-8dee368c1b5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:46:23.548885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea63b3c-d663-451d-86d7-8dee368c1b5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:46:23.550358 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea63b3c-d663-451d-86d7-8dee368c1b5b service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:46:23.561157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1150/2272] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:46:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:46:23.608489 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73792ef-b369-43f3-8302-1a318b42f2b6 service placement] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:46:23.619392 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73792ef-b369-43f3-8302-1a318b42f2b6 service placement] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:46:23.623192 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7808449-b4db-46ee-829a-f20d9c9a4327 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:46:23.623916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7808449-b4db-46ee-829a-f20d9c9a4327 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:46:23.624196 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7808449-b4db-46ee-829a-f20d9c9a4327 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:46:23.624510 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7808449-b4db-46ee-829a-f20d9c9a4327 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:46:23.626361 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d7808449-b4db-46ee-829a-f20d9c9a4327 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:46:23.639874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7808449-b4db-46ee-829a-f20d9c9a4327 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:46:23.641318 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7808449-b4db-46ee-829a-f20d9c9a4327 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:46:23.652493 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1123/2273] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:46:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:46:23.659389 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea63b3c-d663-451d-86d7-8dee368c1b5b service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:46:23.668196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea63b3c-d663-451d-86d7-8dee368c1b5b service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:46:23.671801 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d469923-fb6f-4fa0-81de-1ddb7fb3f8b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:46:23.672438 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d469923-fb6f-4fa0-81de-1ddb7fb3f8b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:46:23.672721 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d469923-fb6f-4fa0-81de-1ddb7fb3f8b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:46:23.672994 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d469923-fb6f-4fa0-81de-1ddb7fb3f8b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:46:23.674841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2d469923-fb6f-4fa0-81de-1ddb7fb3f8b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:46:23.742875 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d469923-fb6f-4fa0-81de-1ddb7fb3f8b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:46:23.744289 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d469923-fb6f-4fa0-81de-1ddb7fb3f8b4 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:46:23.753105 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1151/2274] 10.222.0.20 () {60 vars in 1396 bytes} [Sat Aug 6 12:46:23 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3270 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:46:23.804577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7808449-b4db-46ee-829a-f20d9c9a4327 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:46:23.804577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d7808449-b4db-46ee-829a-f20d9c9a4327 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:46:23.818479 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7808449-b4db-46ee-829a-f20d9c9a4327 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:46:23.824209 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6bc337-e845-4733-bc87-ab9e1c681f7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:46:23.824860 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bc337-e845-4733-bc87-ab9e1c681f7b service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:46:23.825142 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bc337-e845-4733-bc87-ab9e1c681f7b service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:46:23.825419 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bc337-e845-4733-bc87-ab9e1c681f7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:46:23.827299 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-db6bc337-e845-4733-bc87-ab9e1c681f7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:46:23.846796 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d469923-fb6f-4fa0-81de-1ddb7fb3f8b4 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:46:23.849058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2d469923-fb6f-4fa0-81de-1ddb7fb3f8b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:46:23.864821 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d469923-fb6f-4fa0-81de-1ddb7fb3f8b4 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:46:23.870270 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13acffc-aec9-4018-aaf4-cbdc15dbce2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:46:23.870873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13acffc-aec9-4018-aaf4-cbdc15dbce2a service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:46:23.871148 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13acffc-aec9-4018-aaf4-cbdc15dbce2a service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:46:23.871415 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13acffc-aec9-4018-aaf4-cbdc15dbce2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:46:23.873269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d13acffc-aec9-4018-aaf4-cbdc15dbce2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:46:23.901573 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6bc337-e845-4733-bc87-ab9e1c681f7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:46:23.903152 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6bc337-e845-4733-bc87-ab9e1c681f7b service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:46:23.911997 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1124/2275] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:46:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:46:23.945992 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13acffc-aec9-4018-aaf4-cbdc15dbce2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:46:23.947617 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13acffc-aec9-4018-aaf4-cbdc15dbce2a service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:46:23.957319 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1152/2276] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:46:23 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:46:28.568321 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6bc337-e845-4733-bc87-ab9e1c681f7b service nova] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:46:28.569989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-db6bc337-e845-4733-bc87-ab9e1c681f7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:46:28.636285 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6bc337-e845-4733-bc87-ab9e1c681f7b service nova] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:46:28.639876 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106469d4-c836-4c4e-8e6a-dc1ae79c01a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:46:28.640513 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106469d4-c836-4c4e-8e6a-dc1ae79c01a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:46:28.640794 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106469d4-c836-4c4e-8e6a-dc1ae79c01a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:46:28.641067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106469d4-c836-4c4e-8e6a-dc1ae79c01a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:46:28.642893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-106469d4-c836-4c4e-8e6a-dc1ae79c01a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:46:28.652030 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106469d4-c836-4c4e-8e6a-dc1ae79c01a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:46:28.653492 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106469d4-c836-4c4e-8e6a-dc1ae79c01a0 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:46:28.661938 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1125/2277] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:46:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:46:33.952533 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=77 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:46:38.658074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=70 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:49:23.103730 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13acffc-aec9-4018-aaf4-cbdc15dbce2a service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:49:23.142014 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13acffc-aec9-4018-aaf4-cbdc15dbce2a service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:49:23.147780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e4901-6466-435d-8e0d-4f048c6a82e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:49:23.148430 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e4901-6466-435d-8e0d-4f048c6a82e0 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:49:23.148712 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e4901-6466-435d-8e0d-4f048c6a82e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:49:23.148984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e4901-6466-435d-8e0d-4f048c6a82e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:49:23.151103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f14e4901-6466-435d-8e0d-4f048c6a82e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:49:23.228171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e4901-6466-435d-8e0d-4f048c6a82e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:49:23.229714 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e4901-6466-435d-8e0d-4f048c6a82e0 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:49:23.238830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1153/2278] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:49:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:49:23.692621 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106469d4-c836-4c4e-8e6a-dc1ae79c01a0 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:49:23.694344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-106469d4-c836-4c4e-8e6a-dc1ae79c01a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:49:23.754642 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106469d4-c836-4c4e-8e6a-dc1ae79c01a0 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:49:23.758224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4329b9-03a0-4cd1-a337-c8b2a8708893 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:49:23.758825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4329b9-03a0-4cd1-a337-c8b2a8708893 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:49:23.759104 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4329b9-03a0-4cd1-a337-c8b2a8708893 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:49:23.759375 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4329b9-03a0-4cd1-a337-c8b2a8708893 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:49:23.767927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4329b9-03a0-4cd1-a337-c8b2a8708893 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:49:23.769382 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4329b9-03a0-4cd1-a337-c8b2a8708893 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:49:23.777939 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1126/2279] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:49:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:49:24.501422 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e4901-6466-435d-8e0d-4f048c6a82e0 service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:49:24.510556 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e4901-6466-435d-8e0d-4f048c6a82e0 service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:49:24.514612 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e8ee4-5550-4a1c-aa77-b6dd28bda4c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:49:24.515212 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e8ee4-5550-4a1c-aa77-b6dd28bda4c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:49:24.515497 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e8ee4-5550-4a1c-aa77-b6dd28bda4c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:49:24.515771 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e8ee4-5550-4a1c-aa77-b6dd28bda4c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:49:24.517615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4d4e8ee4-5550-4a1c-aa77-b6dd28bda4c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:49:24.540779 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e8ee4-5550-4a1c-aa77-b6dd28bda4c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:49:24.542185 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e8ee4-5550-4a1c-aa77-b6dd28bda4c5 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:49:24.550888 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1154/2280] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:49:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:49:33.774038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=24 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:49:34.546880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=59 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:49:41.137579 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4329b9-03a0-4cd1-a337-c8b2a8708893 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:49:41.167451 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4329b9-03a0-4cd1-a337-c8b2a8708893 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:49:41.173308 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8233a-d138-4cd0-af75-04e08916f937 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2244655ba304b9c980b5e68772eefc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:49:41.173953 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8233a-d138-4cd0-af75-04e08916f937 service placement] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:49:41.174230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8233a-d138-4cd0-af75-04e08916f937 service placement] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:49:41.174503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8233a-d138-4cd0-af75-04e08916f937 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:49:41.183506 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc8233a-d138-4cd0-af75-04e08916f937 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:49:41.185166 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc8233a-d138-4cd0-af75-04e08916f937 service placement] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:49:41.195048 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1127/2281] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:49:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:49:51.190767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=13 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:51:24.044806 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e8ee4-5550-4a1c-aa77-b6dd28bda4c5 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:51:24.046408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4d4e8ee4-5550-4a1c-aa77-b6dd28bda4c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:51:24.067666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e8ee4-5550-4a1c-aa77-b6dd28bda4c5 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:51:24.071488 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1bb95-1e05-459b-b26b-9e421ce33e70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:51:24.072095 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1bb95-1e05-459b-b26b-9e421ce33e70 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:51:24.072409 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1bb95-1e05-459b-b26b-9e421ce33e70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:51:24.072686 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1bb95-1e05-459b-b26b-9e421ce33e70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:51:24.081186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1bb95-1e05-459b-b26b-9e421ce33e70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:51:24.082591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1bb95-1e05-459b-b26b-9e421ce33e70 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:51:24.108666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1155/2282] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:51:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:51:24.171655 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8233a-d138-4cd0-af75-04e08916f937 service placement] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:51:24.200801 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1bb95-1e05-459b-b26b-9e421ce33e70 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:51:24.203778 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8233a-d138-4cd0-af75-04e08916f937 service placement] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:51:24.204965 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1bb95-1e05-459b-b26b-9e421ce33e70 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:51:24.208714 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2c062-21c1-4eae-9ce7-7ad869448c5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:51:24.210309 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2c062-21c1-4eae-9ce7-7ad869448c5d service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:51:24.210442 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab643bd-0c5a-46b0-a7da-35797d32dc03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:51:24.210603 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2c062-21c1-4eae-9ce7-7ad869448c5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:51:24.210889 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2c062-21c1-4eae-9ce7-7ad869448c5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:51:24.211019 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab643bd-0c5a-46b0-a7da-35797d32dc03 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:51:24.211298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab643bd-0c5a-46b0-a7da-35797d32dc03 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:51:24.211570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab643bd-0c5a-46b0-a7da-35797d32dc03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:51:24.219701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab643bd-0c5a-46b0-a7da-35797d32dc03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:51:24.219851 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb2c062-21c1-4eae-9ce7-7ad869448c5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:51:24.221195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb2c062-21c1-4eae-9ce7-7ad869448c5d service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:51:24.221371 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab643bd-0c5a-46b0-a7da-35797d32dc03 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:51:24.248991 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1156/2283] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:51:24 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:51:24.251320 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1128/2284] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:51:24 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:51:28.818815 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2c062-21c1-4eae-9ce7-7ad869448c5d service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:51:28.838055 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2c062-21c1-4eae-9ce7-7ad869448c5d service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:51:28.841675 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16877173-d82b-4d3b-a0fc-9244bbd1d4dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:51:28.842280 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16877173-d82b-4d3b-a0fc-9244bbd1d4dd service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:51:28.842563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16877173-d82b-4d3b-a0fc-9244bbd1d4dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:51:28.842837 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16877173-d82b-4d3b-a0fc-9244bbd1d4dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:51:28.850871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16877173-d82b-4d3b-a0fc-9244bbd1d4dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:51:28.852283 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16877173-d82b-4d3b-a0fc-9244bbd1d4dd service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:51:28.860202 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1157/2285] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:51:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:51:34.246748 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=20 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:51:38.856565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=40 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:52:46.782952 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab643bd-0c5a-46b0-a7da-35797d32dc03 service nova] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:52:46.785453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-eab643bd-0c5a-46b0-a7da-35797d32dc03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:52:46.797301 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab643bd-0c5a-46b0-a7da-35797d32dc03 service nova] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:52:46.801159 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d80a0-f4c4-4272-8021-a02d5c9a16c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:52:46.801761 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d80a0-f4c4-4272-8021-a02d5c9a16c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:52:46.802040 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d80a0-f4c4-4272-8021-a02d5c9a16c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:52:46.802308 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d80a0-f4c4-4272-8021-a02d5c9a16c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:52:46.812815 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d80a0-f4c4-4272-8021-a02d5c9a16c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:52:46.814227 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d80a0-f4c4-4272-8021-a02d5c9a16c2 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:52:46.826455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1129/2286] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:52:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:52:46.904324 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16877173-d82b-4d3b-a0fc-9244bbd1d4dd service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:52:46.913045 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16877173-d82b-4d3b-a0fc-9244bbd1d4dd service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:52:46.916796 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47070f6c-09f1-49d6-b554-0f5657c24a38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:52:46.917704 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47070f6c-09f1-49d6-b554-0f5657c24a38 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:52:46.917704 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47070f6c-09f1-49d6-b554-0f5657c24a38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:52:46.917940 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47070f6c-09f1-49d6-b554-0f5657c24a38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:52:46.919750 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-47070f6c-09f1-49d6-b554-0f5657c24a38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:52:47.014699 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47070f6c-09f1-49d6-b554-0f5657c24a38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:52:47.016973 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47070f6c-09f1-49d6-b554-0f5657c24a38 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:52:47.035976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1158/2287] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:52:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:52:47.805773 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d80a0-f4c4-4272-8021-a02d5c9a16c2 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:52:47.817322 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d80a0-f4c4-4272-8021-a02d5c9a16c2 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:52:47.821414 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98139ea-b749-4d3e-bf1e-c3dcc02b26cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:52:47.822029 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98139ea-b749-4d3e-bf1e-c3dcc02b26cb service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:52:47.822315 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98139ea-b749-4d3e-bf1e-c3dcc02b26cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:52:47.822581 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98139ea-b749-4d3e-bf1e-c3dcc02b26cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:52:47.835547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98139ea-b749-4d3e-bf1e-c3dcc02b26cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:52:47.837255 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98139ea-b749-4d3e-bf1e-c3dcc02b26cb service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:52:47.857284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1130/2288] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:52:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:52:50.979578 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47070f6c-09f1-49d6-b554-0f5657c24a38 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:52:51.008590 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47070f6c-09f1-49d6-b554-0f5657c24a38 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:52:51.014711 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08c774-cde2-446a-9dfe-639229eea6b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:52:51.015936 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08c774-cde2-446a-9dfe-639229eea6b1 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:52:51.016215 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08c774-cde2-446a-9dfe-639229eea6b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:52:51.016539 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08c774-cde2-446a-9dfe-639229eea6b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:52:51.018437 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7f08c774-cde2-446a-9dfe-639229eea6b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:52:51.033424 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f08c774-cde2-446a-9dfe-639229eea6b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:52:51.035380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f08c774-cde2-446a-9dfe-639229eea6b1 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:52:51.048561 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1159/2289] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:52:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:52:51.418889 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98139ea-b749-4d3e-bf1e-c3dcc02b26cb service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:52:51.431130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98139ea-b749-4d3e-bf1e-c3dcc02b26cb service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:52:51.436173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7e8d6-6efc-453e-96bc-ee3953add8c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2244655ba304b9c980b5e68772eefc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:52:51.436879 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7e8d6-6efc-453e-96bc-ee3953add8c5 service placement] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:52:51.437173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7e8d6-6efc-453e-96bc-ee3953add8c5 service placement] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:52:51.437429 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7e8d6-6efc-453e-96bc-ee3953add8c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:52:51.439287 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-30d7e8d6-6efc-453e-96bc-ee3953add8c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:52:51.473065 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7e8d6-6efc-453e-96bc-ee3953add8c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:52:51.474676 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7e8d6-6efc-453e-96bc-ee3953add8c5 service placement] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:52:51.486874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1131/2290] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:52:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:52:54.895975 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08c774-cde2-446a-9dfe-639229eea6b1 service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:52:54.897656 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7f08c774-cde2-446a-9dfe-639229eea6b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:52:54.912051 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08c774-cde2-446a-9dfe-639229eea6b1 service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:52:54.916570 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ad0fd-b315-4fb6-beac-8b4fc1e2e829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:52:54.917191 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ad0fd-b315-4fb6-beac-8b4fc1e2e829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:52:54.917469 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ad0fd-b315-4fb6-beac-8b4fc1e2e829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:52:54.917738 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ad0fd-b315-4fb6-beac-8b4fc1e2e829 admin admin] PATH_INFO: `/v3/users/439c730ea489445998ec6fa480257b5e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:52:54.918869 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ad0fd-b315-4fb6-beac-8b4fc1e2e829 admin admin] RBAC: Authorizing `identity:delete_user(user_id=439c730ea489445998ec6fa480257b5e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:52:54.920790 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ad0fd-b315-4fb6-beac-8b4fc1e2e829 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:52:54.968120 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-190ad0fd-b315-4fb6-beac-8b4fc1e2e829 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '439c730ea489445998ec6fa480257b5e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:52:54.980491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-190ad0fd-b315-4fb6-beac-8b4fc1e2e829 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '439c730ea489445998ec6fa480257b5e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:52:54.981131 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-190ad0fd-b315-4fb6-beac-8b4fc1e2e829 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '439c730ea489445998ec6fa480257b5e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:52:54.981726 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-190ad0fd-b315-4fb6-beac-8b4fc1e2e829 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '439c730ea489445998ec6fa480257b5e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:52:54.989320 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-190ad0fd-b315-4fb6-beac-8b4fc1e2e829 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '439c730ea489445998ec6fa480257b5e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:52:55.002667 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1160/2291] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:52:54 2022] DELETE /identity/v3/users/439c730ea489445998ec6fa480257b5e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:52:55.183124 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7e8d6-6efc-453e-96bc-ee3953add8c5 service placement] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:52:55.194311 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7e8d6-6efc-453e-96bc-ee3953add8c5 service placement] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:52:55.261690 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c9563-a604-4a25-9bb9-c27962e750b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:52:55.262802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c9563-a604-4a25-9bb9-c27962e750b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:52:55.263231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c9563-a604-4a25-9bb9-c27962e750b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:52:55.263652 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c9563-a604-4a25-9bb9-c27962e750b7 admin admin] PATH_INFO: `/v3/projects/c6cb1f1b37a34fedbd0db091e7e830ad` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:52:55.265256 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c9563-a604-4a25-9bb9-c27962e750b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6cb1f1b37a34fedbd0db091e7e830ad)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:52:55.267333 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c9563-a604-4a25-9bb9-c27962e750b7 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:52:55.320519 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c39c9563-a604-4a25-9bb9-c27962e750b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6cb1f1b37a34fedbd0db091e7e830ad', 'request_id': 'req-c39c9563-a604-4a25-9bb9-c27962e750b7'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:52:55.326557 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c39c9563-a604-4a25-9bb9-c27962e750b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6cb1f1b37a34fedbd0db091e7e830ad', 'request_id': 'req-c39c9563-a604-4a25-9bb9-c27962e750b7'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:52:55.334966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c39c9563-a604-4a25-9bb9-c27962e750b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6cb1f1b37a34fedbd0db091e7e830ad', 'request_id': 'req-c39c9563-a604-4a25-9bb9-c27962e750b7'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:52:55.348022 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c39c9563-a604-4a25-9bb9-c27962e750b7 admin admin] The token cache is being invalidate because project c6cb1f1b37a34fedbd0db091e7e830ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:52:55.348404 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c39c9563-a604-4a25-9bb9-c27962e750b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:52:55.349111 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c39c9563-a604-4a25-9bb9-c27962e750b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:52:55.350745 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1132/2292] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:52:55 2022] DELETE /identity/v3/projects/c6cb1f1b37a34fedbd0db091e7e830ad => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:52:55.354682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ad0fd-b315-4fb6-beac-8b4fc1e2e829 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:52:55.356258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-190ad0fd-b315-4fb6-beac-8b4fc1e2e829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:52:55.372745 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ad0fd-b315-4fb6-beac-8b4fc1e2e829 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:52:55.393135 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:52:55.432758 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce677a-7677-4cbe-ad16-1999b601f34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:52:55.433401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce677a-7677-4cbe-ad16-1999b601f34b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:52:55.433684 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce677a-7677-4cbe-ad16-1999b601f34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:52:55.433963 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce677a-7677-4cbe-ad16-1999b601f34b admin admin] PATH_INFO: `/v3/users/942dd1a2c99c4530a72768958e4890d3` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:52:55.447198 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce677a-7677-4cbe-ad16-1999b601f34b admin admin] RBAC: Authorizing `identity:delete_user(user_id=942dd1a2c99c4530a72768958e4890d3)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:52:55.449144 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce677a-7677-4cbe-ad16-1999b601f34b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:52:55.493520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8fce677a-7677-4cbe-ad16-1999b601f34b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '942dd1a2c99c4530a72768958e4890d3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:52:55.505242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8fce677a-7677-4cbe-ad16-1999b601f34b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '942dd1a2c99c4530a72768958e4890d3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:52:55.505895 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8fce677a-7677-4cbe-ad16-1999b601f34b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '942dd1a2c99c4530a72768958e4890d3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:52:55.506506 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8fce677a-7677-4cbe-ad16-1999b601f34b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '942dd1a2c99c4530a72768958e4890d3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:52:55.513899 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8fce677a-7677-4cbe-ad16-1999b601f34b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '942dd1a2c99c4530a72768958e4890d3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:52:55.526482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1161/2293] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 12:52:55 2022] DELETE /identity/v3/users/942dd1a2c99c4530a72768958e4890d3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:52:55.698206 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c9563-a604-4a25-9bb9-c27962e750b7 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:52:55.699912 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c39c9563-a604-4a25-9bb9-c27962e750b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:52:55.715342 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c9563-a604-4a25-9bb9-c27962e750b7 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:52:55.760189 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:52:55.781713 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f9aee-d572-4a24-8965-3786b7df4aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:52:55.782636 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f9aee-d572-4a24-8965-3786b7df4aed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:52:55.782917 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f9aee-d572-4a24-8965-3786b7df4aed admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:52:55.783191 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f9aee-d572-4a24-8965-3786b7df4aed admin admin] PATH_INFO: `/v3/projects/06bd13a5e8654f769cd62b37a7af9639` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:52:55.794207 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f9aee-d572-4a24-8965-3786b7df4aed admin admin] RBAC: Authorizing `identity:delete_project(project_id=06bd13a5e8654f769cd62b37a7af9639)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:52:55.796233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f9aee-d572-4a24-8965-3786b7df4aed admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:52:55.842368 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3f3f9aee-d572-4a24-8965-3786b7df4aed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06bd13a5e8654f769cd62b37a7af9639', 'request_id': 'req-3f3f9aee-d572-4a24-8965-3786b7df4aed'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:52:55.848748 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3f3f9aee-d572-4a24-8965-3786b7df4aed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06bd13a5e8654f769cd62b37a7af9639', 'request_id': 'req-3f3f9aee-d572-4a24-8965-3786b7df4aed'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:52:55.856520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3f3f9aee-d572-4a24-8965-3786b7df4aed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06bd13a5e8654f769cd62b37a7af9639', 'request_id': 'req-3f3f9aee-d572-4a24-8965-3786b7df4aed'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:52:55.868531 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3f3f9aee-d572-4a24-8965-3786b7df4aed admin admin] The token cache is being invalidate because project 06bd13a5e8654f769cd62b37a7af9639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 12:52:55.868817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3f3f9aee-d572-4a24-8965-3786b7df4aed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:52:55.869502 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3f3f9aee-d572-4a24-8965-3786b7df4aed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 12:52:55.871054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1133/2294] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 12:52:55 2022] DELETE /identity/v3/projects/06bd13a5e8654f769cd62b37a7af9639 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 12:53:05.523034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=47,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:53:05.865547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=37,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:54:24.445333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce677a-7677-4cbe-ad16-1999b601f34b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:54:24.446726 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8fce677a-7677-4cbe-ad16-1999b601f34b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:54:24.504169 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce677a-7677-4cbe-ad16-1999b601f34b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:54:24.552294 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27fa45e-e0aa-426e-820c-ab515bb43aeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:54:24.552934 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27fa45e-e0aa-426e-820c-ab515bb43aeb service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:54:24.553211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27fa45e-e0aa-426e-820c-ab515bb43aeb service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:54:24.553480 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27fa45e-e0aa-426e-820c-ab515bb43aeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:54:24.555363 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b27fa45e-e0aa-426e-820c-ab515bb43aeb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:54:24.613888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f9aee-d572-4a24-8965-3786b7df4aed admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:54:24.615499 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3f3f9aee-d572-4a24-8965-3786b7df4aed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:54:24.625254 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27fa45e-e0aa-426e-820c-ab515bb43aeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:54:24.626697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27fa45e-e0aa-426e-820c-ab515bb43aeb service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:54:24.652761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1162/2295] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:54:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:54:24.691639 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f9aee-d572-4a24-8965-3786b7df4aed admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:54:24.697609 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06bbd1-054e-4720-989f-8b182bf83cc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:54:24.698233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06bbd1-054e-4720-989f-8b182bf83cc3 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:54:24.698513 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06bbd1-054e-4720-989f-8b182bf83cc3 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:54:24.698843 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06bbd1-054e-4720-989f-8b182bf83cc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:54:24.700738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fb06bbd1-054e-4720-989f-8b182bf83cc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:54:24.790415 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06bbd1-054e-4720-989f-8b182bf83cc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:54:24.792053 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06bbd1-054e-4720-989f-8b182bf83cc3 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:54:24.810106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1134/2296] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:54:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:54:34.649969 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=76 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:54:34.807420 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=69 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:54:42.118307 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27fa45e-e0aa-426e-820c-ab515bb43aeb service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:54:42.119688 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b27fa45e-e0aa-426e-820c-ab515bb43aeb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:54:42.188362 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27fa45e-e0aa-426e-820c-ab515bb43aeb service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:54:42.193516 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74405e8-a0cf-4018-9565-c0282e186a8d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2244655ba304b9c980b5e68772eefc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:54:42.194129 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74405e8-a0cf-4018-9565-c0282e186a8d service placement] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:54:42.194411 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74405e8-a0cf-4018-9565-c0282e186a8d service placement] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:54:42.194686 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74405e8-a0cf-4018-9565-c0282e186a8d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:54:42.196482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f74405e8-a0cf-4018-9565-c0282e186a8d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:54:42.208759 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74405e8-a0cf-4018-9565-c0282e186a8d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:54:42.210356 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74405e8-a0cf-4018-9565-c0282e186a8d service placement] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:54:42.220640 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1163/2297] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:54:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:54:52.216519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=32 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:56:22.847461 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06bbd1-054e-4720-989f-8b182bf83cc3 service nova] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:56:22.859349 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06bbd1-054e-4720-989f-8b182bf83cc3 service nova] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:56:22.865202 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0ff07-3440-4223-9988-2035bae62800 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:56:22.865806 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0ff07-3440-4223-9988-2035bae62800 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:56:22.866083 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0ff07-3440-4223-9988-2035bae62800 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:56:22.866354 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0ff07-3440-4223-9988-2035bae62800 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:56:22.868066 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-feb0ff07-3440-4223-9988-2035bae62800 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:56:22.924180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb0ff07-3440-4223-9988-2035bae62800 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:56:22.925764 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb0ff07-3440-4223-9988-2035bae62800 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:56:22.935424 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1135/2298] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:56:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:56:23.780048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74405e8-a0cf-4018-9565-c0282e186a8d service placement] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:56:23.791847 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74405e8-a0cf-4018-9565-c0282e186a8d service placement] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:56:23.797189 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b035d-fad0-4600-b0a9-19a81e19f21f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:56:23.797783 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b035d-fad0-4600-b0a9-19a81e19f21f service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:56:23.798059 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b035d-fad0-4600-b0a9-19a81e19f21f service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:56:23.798330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b035d-fad0-4600-b0a9-19a81e19f21f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:56:23.800009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7f1b035d-fad0-4600-b0a9-19a81e19f21f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:56:23.857013 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b035d-fad0-4600-b0a9-19a81e19f21f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:56:23.858596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b035d-fad0-4600-b0a9-19a81e19f21f service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:56:23.867735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1164/2299] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:56:23 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:56:24.893610 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0ff07-3440-4223-9988-2035bae62800 service nova] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:56:24.895043 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-feb0ff07-3440-4223-9988-2035bae62800 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:56:24.907015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0ff07-3440-4223-9988-2035bae62800 service nova] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:56:24.910504 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba9146-0712-49dc-9315-c6143d519a7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:56:24.911097 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba9146-0712-49dc-9315-c6143d519a7b service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:56:24.911372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba9146-0712-49dc-9315-c6143d519a7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:56:24.911643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba9146-0712-49dc-9315-c6143d519a7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:56:24.913313 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a8ba9146-0712-49dc-9315-c6143d519a7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:56:24.981622 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba9146-0712-49dc-9315-c6143d519a7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:56:24.983044 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba9146-0712-49dc-9315-c6143d519a7b service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:56:24.993162 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1136/2300] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:56:24 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:56:33.863652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=28 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:56:34.988108 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=61 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:57:33.839034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b035d-fad0-4600-b0a9-19a81e19f21f service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:57:33.840591 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7f1b035d-fad0-4600-b0a9-19a81e19f21f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:57:33.912454 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b035d-fad0-4600-b0a9-19a81e19f21f service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:57:33.916111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55715c8e-2dcf-4477-928f-3f5a351b03b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:57:33.916787 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55715c8e-2dcf-4477-928f-3f5a351b03b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:57:33.917069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55715c8e-2dcf-4477-928f-3f5a351b03b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:57:33.917345 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55715c8e-2dcf-4477-928f-3f5a351b03b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:57:33.919165 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-55715c8e-2dcf-4477-928f-3f5a351b03b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:57:33.928708 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55715c8e-2dcf-4477-928f-3f5a351b03b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:57:33.930605 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55715c8e-2dcf-4477-928f-3f5a351b03b2 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:57:33.938415 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1165/2301] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:57:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:57:43.934978 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=31 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:57:47.242081 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba9146-0712-49dc-9315-c6143d519a7b service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:57:47.243594 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a8ba9146-0712-49dc-9315-c6143d519a7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:57:47.255722 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba9146-0712-49dc-9315-c6143d519a7b service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:57:47.259251 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d10a9-3197-4ec5-bbb6-1b294cc740a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:57:47.259853 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d10a9-3197-4ec5-bbb6-1b294cc740a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:57:47.260130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d10a9-3197-4ec5-bbb6-1b294cc740a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:57:47.260444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d10a9-3197-4ec5-bbb6-1b294cc740a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:57:47.262162 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-da6d10a9-3197-4ec5-bbb6-1b294cc740a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:57:47.287804 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d10a9-3197-4ec5-bbb6-1b294cc740a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:57:47.290259 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d10a9-3197-4ec5-bbb6-1b294cc740a1 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:57:47.301011 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1137/2302] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:57:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:57:48.229544 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55715c8e-2dcf-4477-928f-3f5a351b03b2 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:57:48.238055 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55715c8e-2dcf-4477-928f-3f5a351b03b2 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:57:48.241620 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8504ded-1170-4451-9f7c-e10bad5410b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:57:48.242719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8504ded-1170-4451-9f7c-e10bad5410b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:57:48.242994 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8504ded-1170-4451-9f7c-e10bad5410b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:57:48.243259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8504ded-1170-4451-9f7c-e10bad5410b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:57:48.244986 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f8504ded-1170-4451-9f7c-e10bad5410b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:57:48.258431 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8504ded-1170-4451-9f7c-e10bad5410b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:57:48.259767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8504ded-1170-4451-9f7c-e10bad5410b2 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:57:48.270308 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1166/2303] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:57:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:57:57.296195 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=11 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:57:58.265630 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=6 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:58:51.228015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7b4a1-63ef-430a-9eb0-4c4f86745cbf None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:58:51.228581 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7b4a1-63ef-430a-9eb0-4c4f86745cbf None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:58:51.228581 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7b4a1-63ef-430a-9eb0-4c4f86745cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:58:51.279437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-e8e7b4a1-63ef-430a-9eb0-4c4f86745cbf None None] MFA Rules not processed for user `257ddc018c33406aa47e7e7129943ae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:58:51.280677 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-e8e7b4a1-63ef-430a-9eb0-4c4f86745cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:58:51.294539 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1138/2304] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:58:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 12:58:51.413206 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1f569-fb91-4b4a-9ce4-37cbc08c6bb7 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:58:51.413719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1f569-fb91-4b4a-9ce4-37cbc08c6bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:58:51.413856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1f569-fb91-4b4a-9ce4-37cbc08c6bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:58:51.480613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8ce1f569-fb91-4b4a-9ce4-37cbc08c6bb7 None None] MFA Rules not processed for user `1125a0b93c9d45ccb2d3f28f188dfd04`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 12:58:51.481895 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8ce1f569-fb91-4b4a-9ce4-37cbc08c6bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:58:51.494567 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1167/2305] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 12:58:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 12:58:51.635380 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7b4a1-63ef-430a-9eb0-4c4f86745cbf None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:58:51.644809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7b4a1-63ef-430a-9eb0-4c4f86745cbf None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:58:51.648977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bac96-326c-4ed4-950a-9a66a859143c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:58:51.649581 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bac96-326c-4ed4-950a-9a66a859143c service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:58:51.649864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bac96-326c-4ed4-950a-9a66a859143c service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:58:51.650139 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8bac96-326c-4ed4-950a-9a66a859143c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:58:51.651887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9f8bac96-326c-4ed4-950a-9a66a859143c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:58:51.661130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8bac96-326c-4ed4-950a-9a66a859143c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:58:51.662541 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8bac96-326c-4ed4-950a-9a66a859143c service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:58:51.670383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1139/2306] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:58:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:58:51.990637 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1f569-fb91-4b4a-9ce4-37cbc08c6bb7 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:58:51.992137 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8ce1f569-fb91-4b4a-9ce4-37cbc08c6bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:58:52.011877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1f569-fb91-4b4a-9ce4-37cbc08c6bb7 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:58:52.017280 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9e96e-785a-4e03-a1a1-d72ce54b6681 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2244655ba304b9c980b5e68772eefc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:58:52.017999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9e96e-785a-4e03-a1a1-d72ce54b6681 service placement] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:58:52.018341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9e96e-785a-4e03-a1a1-d72ce54b6681 service placement] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:58:52.018680 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9e96e-785a-4e03-a1a1-d72ce54b6681 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:58:52.020534 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-48f9e96e-785a-4e03-a1a1-d72ce54b6681 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:58:52.032539 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9e96e-785a-4e03-a1a1-d72ce54b6681 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:58:52.034116 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9e96e-785a-4e03-a1a1-d72ce54b6681 service placement] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:58:52.045257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1168/2307] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:58:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:58:55.226670 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bac96-326c-4ed4-950a-9a66a859143c service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:58:55.229654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9f8bac96-326c-4ed4-950a-9a66a859143c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:58:55.249837 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8bac96-326c-4ed4-950a-9a66a859143c service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:58:55.272691 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cfc35-e6c9-47ad-b8d5-d9eb68fc9a2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:58:55.273626 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cfc35-e6c9-47ad-b8d5-d9eb68fc9a2e service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:58:55.275837 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cfc35-e6c9-47ad-b8d5-d9eb68fc9a2e service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:58:55.276140 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cfc35-e6c9-47ad-b8d5-d9eb68fc9a2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:58:55.282241 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-491cfc35-e6c9-47ad-b8d5-d9eb68fc9a2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:58:55.303190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491cfc35-e6c9-47ad-b8d5-d9eb68fc9a2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:58:55.304908 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491cfc35-e6c9-47ad-b8d5-d9eb68fc9a2e service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:58:55.320067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1140/2308] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:58:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:59:02.040923 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=17 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:59:03.194724 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9e96e-785a-4e03-a1a1-d72ce54b6681 service placement] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:59:03.209229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9e96e-785a-4e03-a1a1-d72ce54b6681 service placement] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:59:03.218660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c087f-5b75-49f0-b786-a3eda2546280 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:59:03.219417 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c087f-5b75-49f0-b786-a3eda2546280 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:59:03.219795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c087f-5b75-49f0-b786-a3eda2546280 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:59:03.220333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c087f-5b75-49f0-b786-a3eda2546280 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:59:03.222248 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f24c087f-5b75-49f0-b786-a3eda2546280 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:59:03.236690 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24c087f-5b75-49f0-b786-a3eda2546280 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:59:03.239798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24c087f-5b75-49f0-b786-a3eda2546280 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:59:03.250762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1169/2309] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:59:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:59:05.313710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=23 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:59:07.858395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cfc35-e6c9-47ad-b8d5-d9eb68fc9a2e service nova] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:59:07.870641 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cfc35-e6c9-47ad-b8d5-d9eb68fc9a2e service nova] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:59:07.876119 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb2123-d4b1-47c0-bfc0-98212d8a7fc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:59:07.876857 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb2123-d4b1-47c0-bfc0-98212d8a7fc2 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:59:07.877199 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb2123-d4b1-47c0-bfc0-98212d8a7fc2 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:59:07.877532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb2123-d4b1-47c0-bfc0-98212d8a7fc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:59:07.879452 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d9bb2123-d4b1-47c0-bfc0-98212d8a7fc2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 12:59:07.894398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb2123-d4b1-47c0-bfc0-98212d8a7fc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:59:07.896057 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb2123-d4b1-47c0-bfc0-98212d8a7fc2 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:59:07.908703 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1141/2310] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:59:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:59:13.246261 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=6 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:59:17.903563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=6 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:59:24.599369 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c087f-5b75-49f0-b786-a3eda2546280 service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:59:24.633936 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c087f-5b75-49f0-b786-a3eda2546280 service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:59:24.639289 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221430a9-04e8-4023-9e42-1b3b2011ec3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:59:24.639884 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221430a9-04e8-4023-9e42-1b3b2011ec3d service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:59:24.640161 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221430a9-04e8-4023-9e42-1b3b2011ec3d service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:59:24.640465 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221430a9-04e8-4023-9e42-1b3b2011ec3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:59:24.648007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221430a9-04e8-4023-9e42-1b3b2011ec3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:59:24.649617 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221430a9-04e8-4023-9e42-1b3b2011ec3d service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:59:24.658216 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1170/2311] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:59:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 12:59:34.654322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=17 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 12:59:49.206912 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb2123-d4b1-47c0-bfc0-98212d8a7fc2 service nova] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 12:59:49.266559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb2123-d4b1-47c0-bfc0-98212d8a7fc2 service nova] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 12:59:49.272402 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972169c-6827-4f1a-b863-3dc40f0b5c29 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2244655ba304b9c980b5e68772eefc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 12:59:49.273017 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972169c-6827-4f1a-b863-3dc40f0b5c29 service placement] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 12:59:49.273295 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972169c-6827-4f1a-b863-3dc40f0b5c29 service placement] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 12:59:49.273565 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972169c-6827-4f1a-b863-3dc40f0b5c29 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 12:59:49.282953 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8972169c-6827-4f1a-b863-3dc40f0b5c29 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 12:59:49.284653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8972169c-6827-4f1a-b863-3dc40f0b5c29 service placement] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 12:59:49.295506 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1142/2312] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 12:59:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 12:59:59.290992 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=27 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:00:25.793294 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221430a9-04e8-4023-9e42-1b3b2011ec3d service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:00:25.803214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221430a9-04e8-4023-9e42-1b3b2011ec3d service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:00:25.807274 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580b193-a206-4c89-a6ca-6ee3d07bf460 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:00:25.807874 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580b193-a206-4c89-a6ca-6ee3d07bf460 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:00:25.808153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580b193-a206-4c89-a6ca-6ee3d07bf460 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:00:25.808498 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580b193-a206-4c89-a6ca-6ee3d07bf460 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:00:25.816711 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e580b193-a206-4c89-a6ca-6ee3d07bf460 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:00:25.818150 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e580b193-a206-4c89-a6ca-6ee3d07bf460 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:00:25.826963 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1171/2313] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:00:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:00:35.822959 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=6 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:01:25.436903 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972169c-6827-4f1a-b863-3dc40f0b5c29 service placement] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:01:25.465376 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972169c-6827-4f1a-b863-3dc40f0b5c29 service placement] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:01:25.471016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf44d27-a7bd-4e85-8d89-8d382c256c12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:01:25.471648 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf44d27-a7bd-4e85-8d89-8d382c256c12 service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:01:25.471951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf44d27-a7bd-4e85-8d89-8d382c256c12 service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:01:25.472225 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf44d27-a7bd-4e85-8d89-8d382c256c12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:01:25.480874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf44d27-a7bd-4e85-8d89-8d382c256c12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:01:25.482474 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf44d27-a7bd-4e85-8d89-8d382c256c12 service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:01:25.509084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1143/2314] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:01:25 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:01:35.506531 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=20 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:02:27.727906 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ad15e-a94e-4497-8bba-0e0ca2dc4ad1 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:02:27.728473 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ad15e-a94e-4497-8bba-0e0ca2dc4ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:02:27.728473 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ad15e-a94e-4497-8bba-0e0ca2dc4ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:02:27.777227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1c8ad15e-a94e-4497-8bba-0e0ca2dc4ad1 None None] MFA Rules not processed for user `6844f315435f48f68488a2f0459bf155`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:02:27.778418 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1c8ad15e-a94e-4497-8bba-0e0ca2dc4ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:02:27.785172 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1172/2315] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:02:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:02:28.091577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf44d27-a7bd-4e85-8d89-8d382c256c12 service nova] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:02:28.111056 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf44d27-a7bd-4e85-8d89-8d382c256c12 service nova] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:02:28.114665 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4ef0b-ed7e-46af-8518-3b8094409d48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:02:28.115265 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4ef0b-ed7e-46af-8518-3b8094409d48 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:02:28.115546 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4ef0b-ed7e-46af-8518-3b8094409d48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:02:28.115820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4ef0b-ed7e-46af-8518-3b8094409d48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:02:28.117591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-94b4ef0b-ed7e-46af-8518-3b8094409d48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:02:28.126861 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4ef0b-ed7e-46af-8518-3b8094409d48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:02:28.128279 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4ef0b-ed7e-46af-8518-3b8094409d48 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:02:28.136088 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1144/2316] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:02:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 13:02:33.950126 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ad15e-a94e-4497-8bba-0e0ca2dc4ad1 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:02:33.961003 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ad15e-a94e-4497-8bba-0e0ca2dc4ad1 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:02:33.964713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26588b9-05f0-46e8-aa36-ef8d9ff1af9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:02:33.965334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26588b9-05f0-46e8-aa36-ef8d9ff1af9f service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:02:33.965616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26588b9-05f0-46e8-aa36-ef8d9ff1af9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:02:33.965874 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26588b9-05f0-46e8-aa36-ef8d9ff1af9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:02:33.973683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26588b9-05f0-46e8-aa36-ef8d9ff1af9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:02:33.975708 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26588b9-05f0-46e8-aa36-ef8d9ff1af9f service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:02:33.984321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1173/2317] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:02:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:02:38.132776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=10 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:02:43.980839 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=21 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:02:47.948239 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4ef0b-ed7e-46af-8518-3b8094409d48 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:02:47.956646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4ef0b-ed7e-46af-8518-3b8094409d48 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:02:47.961056 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6bda70-cfc0-4780-b62f-8d35800f038c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:02:47.961663 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bda70-cfc0-4780-b62f-8d35800f038c service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:02:47.961940 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bda70-cfc0-4780-b62f-8d35800f038c service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:02:47.962212 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bda70-cfc0-4780-b62f-8d35800f038c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:02:47.972293 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6bda70-cfc0-4780-b62f-8d35800f038c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:02:47.973740 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6bda70-cfc0-4780-b62f-8d35800f038c service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:02:47.984787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1145/2318] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:02:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:02:49.103914 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26588b9-05f0-46e8-aa36-ef8d9ff1af9f service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:02:49.112555 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26588b9-05f0-46e8-aa36-ef8d9ff1af9f service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:02:49.116126 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8002b418-fe33-44ca-aa80-b21b429d5b3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:02:49.116755 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002b418-fe33-44ca-aa80-b21b429d5b3c service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:02:49.117033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002b418-fe33-44ca-aa80-b21b429d5b3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:02:49.117306 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002b418-fe33-44ca-aa80-b21b429d5b3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:02:49.127232 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8002b418-fe33-44ca-aa80-b21b429d5b3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:02:49.128674 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8002b418-fe33-44ca-aa80-b21b429d5b3c service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:02:49.139548 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1174/2319] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:02:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:02:52.067529 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6bda70-cfc0-4780-b62f-8d35800f038c service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:02:52.068951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-9d6bda70-cfc0-4780-b62f-8d35800f038c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:02:52.102862 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6bda70-cfc0-4780-b62f-8d35800f038c service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:02:52.106601 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e5f76-29a6-41ff-b518-3d1b8360da1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:02:52.107201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e5f76-29a6-41ff-b518-3d1b8360da1d service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:02:52.107475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e5f76-29a6-41ff-b518-3d1b8360da1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:02:52.107742 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e5f76-29a6-41ff-b518-3d1b8360da1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:02:52.116201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e5f76-29a6-41ff-b518-3d1b8360da1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:02:52.117646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e5f76-29a6-41ff-b518-3d1b8360da1d service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:02:52.147084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1146/2320] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:02:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:02:52.190382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8002b418-fe33-44ca-aa80-b21b429d5b3c service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:02:52.199892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8002b418-fe33-44ca-aa80-b21b429d5b3c service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:02:52.203411 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0ace4-48e9-48bd-a812-9859d1b6cbe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:02:52.203997 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0ace4-48e9-48bd-a812-9859d1b6cbe3 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:02:52.204274 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0ace4-48e9-48bd-a812-9859d1b6cbe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:02:52.204574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0ace4-48e9-48bd-a812-9859d1b6cbe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:02:52.213392 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd0ace4-48e9-48bd-a812-9859d1b6cbe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:02:52.214817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd0ace4-48e9-48bd-a812-9859d1b6cbe3 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:02:52.241107 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1175/2321] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:02:52 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:03:02.143073 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=28 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:03:02.238610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=19 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:04:52.577701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e5f76-29a6-41ff-b518-3d1b8360da1d service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:04:52.631681 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e5f76-29a6-41ff-b518-3d1b8360da1d service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:04:52.635484 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4336d-08db-41f3-8964-11ecf782be50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:04:52.636099 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4336d-08db-41f3-8964-11ecf782be50 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:04:52.636407 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4336d-08db-41f3-8964-11ecf782be50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:04:52.636678 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4336d-08db-41f3-8964-11ecf782be50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:04:52.638537 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3eb4336d-08db-41f3-8964-11ecf782be50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:04:52.707579 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4336d-08db-41f3-8964-11ecf782be50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:04:52.709042 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4336d-08db-41f3-8964-11ecf782be50 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:04:52.727147 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1147/2322] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:04:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:04:53.403017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0ace4-48e9-48bd-a812-9859d1b6cbe3 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:04:53.404661 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ecd0ace4-48e9-48bd-a812-9859d1b6cbe3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:04:53.480681 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0ace4-48e9-48bd-a812-9859d1b6cbe3 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:04:53.486201 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda1aca-0020-41c5-b912-86da9d8c4431 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:04:53.486807 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda1aca-0020-41c5-b912-86da9d8c4431 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:04:53.487083 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda1aca-0020-41c5-b912-86da9d8c4431 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:04:53.487353 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda1aca-0020-41c5-b912-86da9d8c4431 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:04:53.496136 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda1aca-0020-41c5-b912-86da9d8c4431 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:04:53.497750 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda1aca-0020-41c5-b912-86da9d8c4431 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:04:53.509113 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1176/2323] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:04:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:04:55.175934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4336d-08db-41f3-8964-11ecf782be50 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:04:55.177503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3eb4336d-08db-41f3-8964-11ecf782be50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:04:55.256072 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4336d-08db-41f3-8964-11ecf782be50 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:04:55.260906 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732035e-d6b5-4043-bbd9-58ef017fe617 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2244655ba304b9c980b5e68772eefc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:04:55.261509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732035e-d6b5-4043-bbd9-58ef017fe617 service placement] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:04:55.261785 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732035e-d6b5-4043-bbd9-58ef017fe617 service placement] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:04:55.262055 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732035e-d6b5-4043-bbd9-58ef017fe617 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:04:55.263837 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d732035e-d6b5-4043-bbd9-58ef017fe617 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:04:55.275372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732035e-d6b5-4043-bbd9-58ef017fe617 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:04:55.277011 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732035e-d6b5-4043-bbd9-58ef017fe617 service placement] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:04:55.286596 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1148/2324] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:04:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:05:03.504898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=34 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:05:05.283110 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=93 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:06:23.129207 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda1aca-0020-41c5-b912-86da9d8c4431 service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:06:23.139563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda1aca-0020-41c5-b912-86da9d8c4431 service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:06:23.143969 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d902bf-033c-4467-b043-c1c371776bf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:06:23.144608 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d902bf-033c-4467-b043-c1c371776bf8 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:23.144889 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d902bf-033c-4467-b043-c1c371776bf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:23.145164 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d902bf-033c-4467-b043-c1c371776bf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:23.147021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-31d902bf-033c-4467-b043-c1c371776bf8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:06:23.245756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d902bf-033c-4467-b043-c1c371776bf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:06:23.247228 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d902bf-033c-4467-b043-c1c371776bf8 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:06:23.266087 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1177/2325] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:06:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:06:25.236722 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732035e-d6b5-4043-bbd9-58ef017fe617 service placement] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:06:25.247006 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732035e-d6b5-4043-bbd9-58ef017fe617 service placement] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:06:25.250719 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3e768-4483-4f6b-bfb8-27a6643aba06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:06:25.251341 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3e768-4483-4f6b-bfb8-27a6643aba06 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:25.251650 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3e768-4483-4f6b-bfb8-27a6643aba06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:25.251919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3e768-4483-4f6b-bfb8-27a6643aba06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:25.254967 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a9a3e768-4483-4f6b-bfb8-27a6643aba06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:06:25.333145 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3e768-4483-4f6b-bfb8-27a6643aba06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:06:25.334600 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3e768-4483-4f6b-bfb8-27a6643aba06 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:06:25.343984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1149/2326] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:06:25 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 13:06:26.018161 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d902bf-033c-4467-b043-c1c371776bf8 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:06:26.027497 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d902bf-033c-4467-b043-c1c371776bf8 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:06:26.031179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0b866-7461-4a46-a56b-51c6e4dad45c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:06:26.031787 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0b866-7461-4a46-a56b-51c6e4dad45c service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:26.032068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0b866-7461-4a46-a56b-51c6e4dad45c service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:26.032376 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0b866-7461-4a46-a56b-51c6e4dad45c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:26.034104 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0fc0b866-7461-4a46-a56b-51c6e4dad45c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:06:26.123998 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc0b866-7461-4a46-a56b-51c6e4dad45c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:06:26.125877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc0b866-7461-4a46-a56b-51c6e4dad45c service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:06:26.145599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1178/2327] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:06:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:06:26.292822 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ed4b0-2835-4c69-838e-fdcc2b86ccca None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:26.293287 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ed4b0-2835-4c69-838e-fdcc2b86ccca None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:26.293389 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ed4b0-2835-4c69-838e-fdcc2b86ccca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:26.359959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-5d9ed4b0-2835-4c69-838e-fdcc2b86ccca None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:06:26.361306 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5d9ed4b0-2835-4c69-838e-fdcc2b86ccca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:06:26.370106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1150/2328] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:06:26.379322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6be12-e4de-4dea-85b4-7e0a73d0a8b6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:26.380784 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6be12-e4de-4dea-85b4-7e0a73d0a8b6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:26.381246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6be12-e4de-4dea-85b4-7e0a73d0a8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:26.403238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-84a6be12-e4de-4dea-85b4-7e0a73d0a8b6 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:06:26.404497 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-84a6be12-e4de-4dea-85b4-7e0a73d0a8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:06:26.412711 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1179/2329] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 13:06:26.416927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ed4b0-2835-4c69-838e-fdcc2b86ccca None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:06:26.427379 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ed4b0-2835-4c69-838e-fdcc2b86ccca None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:06:26.432384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8572b94-ac82-4752-84f0-5eb8be237bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:06:26.433002 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8572b94-ac82-4752-84f0-5eb8be237bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:26.433278 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8572b94-ac82-4752-84f0-5eb8be237bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:26.433548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8572b94-ac82-4752-84f0-5eb8be237bd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:26.434234 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8572b94-ac82-4752-84f0-5eb8be237bd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:06:26.434535 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8572b94-ac82-4752-84f0-5eb8be237bd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:06:26.436121 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8572b94-ac82-4752-84f0-5eb8be237bd0 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:06:26.449450 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1151/2330] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 13:06:26.453734 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6be12-e4de-4dea-85b4-7e0a73d0a8b6 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:06:26.462917 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6be12-e4de-4dea-85b4-7e0a73d0a8b6 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:06:26.467482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12deed7c-3ac1-460f-8a01-8f7a51cfeca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:06:26.468089 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12deed7c-3ac1-460f-8a01-8f7a51cfeca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:26.468429 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12deed7c-3ac1-460f-8a01-8f7a51cfeca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:26.468704 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12deed7c-3ac1-460f-8a01-8f7a51cfeca6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:26.469382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12deed7c-3ac1-460f-8a01-8f7a51cfeca6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:06:26.471270 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12deed7c-3ac1-460f-8a01-8f7a51cfeca6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:06:26.507081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:06:26.521160 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1180/2331] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:06:26 2022] POST /identity/v3/projects => generated 381 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:06:26.522336 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8572b94-ac82-4752-84f0-5eb8be237bd0 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:06:26.532832 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8572b94-ac82-4752-84f0-5eb8be237bd0 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:06:26.568237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12deed7c-3ac1-460f-8a01-8f7a51cfeca6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:06:26.608400 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61162f2-62a2-4164-9c0e-2a92534b7e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:06:26.609655 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61162f2-62a2-4164-9c0e-2a92534b7e27 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:26.610018 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61162f2-62a2-4164-9c0e-2a92534b7e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:26.610296 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61162f2-62a2-4164-9c0e-2a92534b7e27 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:26.611723 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61162f2-62a2-4164-9c0e-2a92534b7e27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:06:26.613622 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61162f2-62a2-4164-9c0e-2a92534b7e27 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:06:26.634965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1152/2332] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:06:26 2022] POST /identity/v3/users => generated 291 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:06:26.638992 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61162f2-62a2-4164-9c0e-2a92534b7e27 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:06:26.648495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61162f2-62a2-4164-9c0e-2a92534b7e27 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:06:26.653098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee8f2c-2da1-4c91-bad0-3ffe3db61975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:06:26.653693 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee8f2c-2da1-4c91-bad0-3ffe3db61975 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:26.653835 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12deed7c-3ac1-460f-8a01-8f7a51cfeca6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:06:26.654038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee8f2c-2da1-4c91-bad0-3ffe3db61975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:26.654245 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee8f2c-2da1-4c91-bad0-3ffe3db61975 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:26.654929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee8f2c-2da1-4c91-bad0-3ffe3db61975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:06:26.657094 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee8f2c-2da1-4c91-bad0-3ffe3db61975 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:06:26.660277 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3fe2b-e232-41db-a17a-340f2d2feeb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:06:26.660915 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3fe2b-e232-41db-a17a-340f2d2feeb6 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:26.661196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3fe2b-e232-41db-a17a-340f2d2feeb6 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:26.661470 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3fe2b-e232-41db-a17a-340f2d2feeb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:26.672037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1153/2333] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 13:06:26.672667 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3fe2b-e232-41db-a17a-340f2d2feeb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:06:26.673524 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee8f2c-2da1-4c91-bad0-3ffe3db61975 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:06:26.674398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3fe2b-e232-41db-a17a-340f2d2feeb6 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:06:26.683879 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee8f2c-2da1-4c91-bad0-3ffe3db61975 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:06:26.685494 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1181/2334] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:06:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 13:06:26.690198 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6b49f-2bcb-4a86-8ba2-21db56e0d236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:06:26.691041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6b49f-2bcb-4a86-8ba2-21db56e0d236 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:26.691627 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6b49f-2bcb-4a86-8ba2-21db56e0d236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:26.692188 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6b49f-2bcb-4a86-8ba2-21db56e0d236 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:26.692983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6b49f-2bcb-4a86-8ba2-21db56e0d236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:06:26.694756 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6b49f-2bcb-4a86-8ba2-21db56e0d236 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:06:26.706943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1154/2335] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 13:06:26.711114 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3fe2b-e232-41db-a17a-340f2d2feeb6 service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:06:26.723451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3fe2b-e232-41db-a17a-340f2d2feeb6 service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:06:26.730985 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaebf41-2d26-4042-bc6e-3d27025d5028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:06:26.731667 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaebf41-2d26-4042-bc6e-3d27025d5028 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:26.731953 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaebf41-2d26-4042-bc6e-3d27025d5028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:26.732240 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaebf41-2d26-4042-bc6e-3d27025d5028 admin admin] PATH_INFO: `/v3/projects/f7477fc540f843b4936879b9425896f3/users/3188163f95d04235b5de120354ea4997/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:26.748444 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaebf41-2d26-4042-bc6e-3d27025d5028 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7477fc540f843b4936879b9425896f3, user_id=3188163f95d04235b5de120354ea4997, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:06:26.751432 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaebf41-2d26-4042-bc6e-3d27025d5028 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:06:26.765130 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1182/2336] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:06:26 2022] PUT /identity/v3/projects/f7477fc540f843b4936879b9425896f3/users/3188163f95d04235b5de120354ea4997/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:06:26.770781 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30dacc-e035-4872-9a12-f1a78362de08 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:26.771086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30dacc-e035-4872-9a12-f1a78362de08 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:26.771359 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30dacc-e035-4872-9a12-f1a78362de08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:26.779332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:06:26.792001 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-bb30dacc-e035-4872-9a12-f1a78362de08 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:06:26.793231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-bb30dacc-e035-4872-9a12-f1a78362de08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:06:26.865302 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1155/2337] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:06:26.880225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaebf41-2d26-4042-bc6e-3d27025d5028 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:06:26.954495 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaebf41-2d26-4042-bc6e-3d27025d5028 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:06:26.958865 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65040fb0-c4db-4de0-8547-1b37308d6306 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:06:26.959936 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65040fb0-c4db-4de0-8547-1b37308d6306 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:26.960193 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65040fb0-c4db-4de0-8547-1b37308d6306 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:26.960568 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65040fb0-c4db-4de0-8547-1b37308d6306 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:26.978293 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65040fb0-c4db-4de0-8547-1b37308d6306 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:06:26.979740 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65040fb0-c4db-4de0-8547-1b37308d6306 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:06:26.991280 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1183/2338] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:06:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:06:32.414372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb863442-420d-4f4f-8a9a-04fab38c0f4c None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:32.414900 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb863442-420d-4f4f-8a9a-04fab38c0f4c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:32.414900 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb863442-420d-4f4f-8a9a-04fab38c0f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:32.442941 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-eb863442-420d-4f4f-8a9a-04fab38c0f4c None None] MFA Rules not processed for user `3188163f95d04235b5de120354ea4997`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:06:32.444072 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-eb863442-420d-4f4f-8a9a-04fab38c0f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:06:32.507254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1156/2339] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:06:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:06:32.511682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65040fb0-c4db-4de0-8547-1b37308d6306 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:06:32.521121 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65040fb0-c4db-4de0-8547-1b37308d6306 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:06:32.525642 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfb6d4-68ae-4475-af00-94311d82608b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:06:32.526229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfb6d4-68ae-4475-af00-94311d82608b admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:32.526502 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfb6d4-68ae-4475-af00-94311d82608b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:32.526770 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfb6d4-68ae-4475-af00-94311d82608b admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:32.527423 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbfb6d4-68ae-4475-af00-94311d82608b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:06:32.529334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbfb6d4-68ae-4475-af00-94311d82608b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:06:32.558657 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1184/2340] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:06:32 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:06:32.559699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb863442-420d-4f4f-8a9a-04fab38c0f4c None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:06:32.571158 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb863442-420d-4f4f-8a9a-04fab38c0f4c None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:06:32.627234 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db550e-4dd4-47d0-8bf0-7073a8786936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:06:32.627852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db550e-4dd4-47d0-8bf0-7073a8786936 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:32.628162 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db550e-4dd4-47d0-8bf0-7073a8786936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:32.628474 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db550e-4dd4-47d0-8bf0-7073a8786936 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:32.629181 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db550e-4dd4-47d0-8bf0-7073a8786936 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:06:32.631113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db550e-4dd4-47d0-8bf0-7073a8786936 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:06:32.652141 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1157/2341] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:06:32 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:06:32.656632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfb6d4-68ae-4475-af00-94311d82608b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:06:32.667018 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfb6d4-68ae-4475-af00-94311d82608b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:06:32.672293 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7deed-964a-46fc-b242-63d0080dc8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:06:32.672918 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7deed-964a-46fc-b242-63d0080dc8fb admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:32.673198 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7deed-964a-46fc-b242-63d0080dc8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:32.673469 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7deed-964a-46fc-b242-63d0080dc8fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:32.674158 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7deed-964a-46fc-b242-63d0080dc8fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:06:32.675729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7deed-964a-46fc-b242-63d0080dc8fb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:06:32.687079 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1185/2342] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:06:32.691000 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db550e-4dd4-47d0-8bf0-7073a8786936 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:06:32.700921 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db550e-4dd4-47d0-8bf0-7073a8786936 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:06:32.705469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca842b6-de94-4d8a-9628-00554675839c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:06:32.706118 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca842b6-de94-4d8a-9628-00554675839c admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:32.706396 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca842b6-de94-4d8a-9628-00554675839c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:32.706778 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca842b6-de94-4d8a-9628-00554675839c admin admin] PATH_INFO: `/v3/projects/257ddc6f37a64b71b6eb8ce15fc3e79f/users/0b67f99b002c4879b006efb048c5aaf4/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:32.722075 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca842b6-de94-4d8a-9628-00554675839c admin admin] RBAC: Authorizing `identity:create_grant(project_id=257ddc6f37a64b71b6eb8ce15fc3e79f, user_id=0b67f99b002c4879b006efb048c5aaf4, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:06:32.724747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca842b6-de94-4d8a-9628-00554675839c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:06:32.734376 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:06:32.739913 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1158/2343] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:06:32 2022] PUT /identity/v3/projects/257ddc6f37a64b71b6eb8ce15fc3e79f/users/0b67f99b002c4879b006efb048c5aaf4/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:06:32.743688 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7deed-964a-46fc-b242-63d0080dc8fb admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:06:32.753407 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7deed-964a-46fc-b242-63d0080dc8fb admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:06:32.778542 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:06:32.813529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84e0d6-f82e-4fdf-81ab-43833fc7c42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:06:32.814238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84e0d6-f82e-4fdf-81ab-43833fc7c42c admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:32.814364 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84e0d6-f82e-4fdf-81ab-43833fc7c42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:32.814765 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84e0d6-f82e-4fdf-81ab-43833fc7c42c admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:32.815489 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84e0d6-f82e-4fdf-81ab-43833fc7c42c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:06:32.817242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84e0d6-f82e-4fdf-81ab-43833fc7c42c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:06:32.830989 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1186/2344] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:06:32.831960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca842b6-de94-4d8a-9628-00554675839c admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:06:32.844341 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca842b6-de94-4d8a-9628-00554675839c admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:06:32.851138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cbf69c-dd6d-43f7-9b22-3f71e7e244fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:06:32.852056 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbf69c-dd6d-43f7-9b22-3f71e7e244fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:32.864560 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbf69c-dd6d-43f7-9b22-3f71e7e244fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:32.864560 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbf69c-dd6d-43f7-9b22-3f71e7e244fc admin admin] PATH_INFO: `/v3/projects/257ddc6f37a64b71b6eb8ce15fc3e79f/users/0b67f99b002c4879b006efb048c5aaf4/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:32.864560 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cbf69c-dd6d-43f7-9b22-3f71e7e244fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=257ddc6f37a64b71b6eb8ce15fc3e79f, user_id=0b67f99b002c4879b006efb048c5aaf4, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:06:32.864560 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cbf69c-dd6d-43f7-9b22-3f71e7e244fc admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:06:32.869285 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1159/2345] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:06:32 2022] PUT /identity/v3/projects/257ddc6f37a64b71b6eb8ce15fc3e79f/users/0b67f99b002c4879b006efb048c5aaf4/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:06:37.753247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c21a1b-a456-4b6c-9dae-684f43a4ae6a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:37.753803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c21a1b-a456-4b6c-9dae-684f43a4ae6a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:37.753912 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c21a1b-a456-4b6c-9dae-684f43a4ae6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:37.789590 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-49c21a1b-a456-4b6c-9dae-684f43a4ae6a None None] MFA Rules not processed for user `0b67f99b002c4879b006efb048c5aaf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:06:37.790916 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-49c21a1b-a456-4b6c-9dae-684f43a4ae6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:06:37.919018 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1187/2346] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:06:37 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:06:37.931497 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cbf69c-dd6d-43f7-9b22-3f71e7e244fc admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:06:37.933254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-26cbf69c-dd6d-43f7-9b22-3f71e7e244fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:06:38.057045 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cbf69c-dd6d-43f7-9b22-3f71e7e244fc admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:06:38.061076 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0293e7e-4882-405e-8b4f-4f79500edf76 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7610d6289efb44599d0cee03b93f9938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:06:38.061741 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0293e7e-4882-405e-8b4f-4f79500edf76 service glance] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:38.062176 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0293e7e-4882-405e-8b4f-4f79500edf76 service glance] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:38.062445 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0293e7e-4882-405e-8b4f-4f79500edf76 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:38.128751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0293e7e-4882-405e-8b4f-4f79500edf76 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:06:38.130475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0293e7e-4882-405e-8b4f-4f79500edf76 service glance] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:06:38.142009 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1160/2347] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:06:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:06:38.278630 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68862f-148e-4cfe-b6aa-40157e328c42 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:38.278925 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68862f-148e-4cfe-b6aa-40157e328c42 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:38.279204 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68862f-148e-4cfe-b6aa-40157e328c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:38.341719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8a68862f-148e-4cfe-b6aa-40157e328c42 None None] MFA Rules not processed for user `c810b52785b94c0cab4fd9f0d6e65825`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:06:38.343635 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8a68862f-148e-4cfe-b6aa-40157e328c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:06:38.414328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1188/2348] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Aug 6 13:06:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:06:38.460694 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0293e7e-4882-405e-8b4f-4f79500edf76 service glance] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:06:38.462095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f0293e7e-4882-405e-8b4f-4f79500edf76 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:06:38.554887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:06:38.557953 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0293e7e-4882-405e-8b4f-4f79500edf76 service glance] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:06:38.564506 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d857f61d-f24c-4a62-87e2-11938f7354a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efc792a2655439f8b8070010059b6bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:06:38.565126 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857f61d-f24c-4a62-87e2-11938f7354a3 service swift] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:38.565409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857f61d-f24c-4a62-87e2-11938f7354a3 service swift] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:38.565786 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857f61d-f24c-4a62-87e2-11938f7354a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:38.574305 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d857f61d-f24c-4a62-87e2-11938f7354a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:06:38.575972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d857f61d-f24c-4a62-87e2-11938f7354a3 service swift] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:06:38.585186 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1161/2349] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Aug 6 13:06:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 13:06:45.209674 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68862f-148e-4cfe-b6aa-40157e328c42 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:06:45.226150 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:06:45.308683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68862f-148e-4cfe-b6aa-40157e328c42 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:06:45.318300 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0dd72-b3ac-4b5c-a64d-4da7ffe2ee61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:06:45.318735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0dd72-b3ac-4b5c-a64d-4da7ffe2ee61 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:45.319080 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0dd72-b3ac-4b5c-a64d-4da7ffe2ee61 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:45.319572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0dd72-b3ac-4b5c-a64d-4da7ffe2ee61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:45.329783 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0dd72-b3ac-4b5c-a64d-4da7ffe2ee61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:06:45.331589 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0dd72-b3ac-4b5c-a64d-4da7ffe2ee61 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:06:45.347118 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1189/2350] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:06:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:06:48.316940 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc87cf-8a48-4dc2-8729-c33741725a42 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:48.316940 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc87cf-8a48-4dc2-8729-c33741725a42 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:48.317569 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc87cf-8a48-4dc2-8729-c33741725a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:48.342684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-6dcc87cf-8a48-4dc2-8729-c33741725a42 None None] MFA Rules not processed for user `c810b52785b94c0cab4fd9f0d6e65825`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:06:48.344444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6dcc87cf-8a48-4dc2-8729-c33741725a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:06:48.359038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1162/2351] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Aug 6 13:06:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:06:48.382650 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0dd72-b3ac-4b5c-a64d-4da7ffe2ee61 service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:06:48.395097 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0dd72-b3ac-4b5c-a64d-4da7ffe2ee61 service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:06:48.401306 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b6893-d3df-4f63-85c8-e65fa8689485 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efc792a2655439f8b8070010059b6bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:06:48.402170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b6893-d3df-4f63-85c8-e65fa8689485 service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:48.404659 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b6893-d3df-4f63-85c8-e65fa8689485 service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:48.405041 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b6893-d3df-4f63-85c8-e65fa8689485 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:48.414613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b6893-d3df-4f63-85c8-e65fa8689485 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:06:48.418805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b6893-d3df-4f63-85c8-e65fa8689485 service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:06:48.437571 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1190/2352] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Aug 6 13:06:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 13:06:48.873081 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f6595-76ec-40b0-b5fb-c3b4011df5d0 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:48.874366 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f6595-76ec-40b0-b5fb-c3b4011df5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:48.874805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f6595-76ec-40b0-b5fb-c3b4011df5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:48.951078 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-726f6595-76ec-40b0-b5fb-c3b4011df5d0 None None] MFA Rules not processed for user `3188163f95d04235b5de120354ea4997`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:06:48.952517 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-726f6595-76ec-40b0-b5fb-c3b4011df5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:06:48.973883 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1163/2353] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:06:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:06:49.633246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b6893-d3df-4f63-85c8-e65fa8689485 service swift] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:06:49.746123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b6893-d3df-4f63-85c8-e65fa8689485 service swift] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:06:49.754208 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1092721-fb13-41f1-9782-45c8c8150b18 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2244655ba304b9c980b5e68772eefc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:06:49.754897 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1092721-fb13-41f1-9782-45c8c8150b18 service placement] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:49.755182 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1092721-fb13-41f1-9782-45c8c8150b18 service placement] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:49.755463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1092721-fb13-41f1-9782-45c8c8150b18 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:49.766893 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1092721-fb13-41f1-9782-45c8c8150b18 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:06:49.768576 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1092721-fb13-41f1-9782-45c8c8150b18 service placement] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:06:49.800109 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1191/2354] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:06:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:06:50.356182 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c8fd2-7deb-4736-a2f3-bfe386396ac8 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:50.356698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c8fd2-7deb-4736-a2f3-bfe386396ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:50.356799 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c8fd2-7deb-4736-a2f3-bfe386396ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:50.384762 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-0e6c8fd2-7deb-4736-a2f3-bfe386396ac8 None None] MFA Rules not processed for user `1125a0b93c9d45ccb2d3f28f188dfd04`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:06:50.386723 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0e6c8fd2-7deb-4736-a2f3-bfe386396ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:06:50.509015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1164/2355] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:06:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:06:50.940137 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1092721-fb13-41f1-9782-45c8c8150b18 service placement] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:06:51.012525 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1092721-fb13-41f1-9782-45c8c8150b18 service placement] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:06:51.017687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147d65f-cbdb-431a-b2a3-b8f41fca1e82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:06:51.018393 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147d65f-cbdb-431a-b2a3-b8f41fca1e82 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:51.018790 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147d65f-cbdb-431a-b2a3-b8f41fca1e82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:51.019244 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147d65f-cbdb-431a-b2a3-b8f41fca1e82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:51.027626 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4147d65f-cbdb-431a-b2a3-b8f41fca1e82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:06:51.029184 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4147d65f-cbdb-431a-b2a3-b8f41fca1e82 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:06:51.048827 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1192/2356] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:06:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:06:52.725241 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59863d85-ad3a-4a3a-81d7-050eaa7b8a81 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:52.726173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59863d85-ad3a-4a3a-81d7-050eaa7b8a81 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:52.726934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59863d85-ad3a-4a3a-81d7-050eaa7b8a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:52.747798 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-59863d85-ad3a-4a3a-81d7-050eaa7b8a81 None None] MFA Rules not processed for user `c810b52785b94c0cab4fd9f0d6e65825`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:06:52.750085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-59863d85-ad3a-4a3a-81d7-050eaa7b8a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:06:52.758190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1165/2357] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Aug 6 13:06:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:06:52.775406 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147d65f-cbdb-431a-b2a3-b8f41fca1e82 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:06:52.792641 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147d65f-cbdb-431a-b2a3-b8f41fca1e82 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:06:52.798932 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f726b88-dab0-42ac-8239-c6ee1f0101f1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efc792a2655439f8b8070010059b6bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:06:52.799977 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f726b88-dab0-42ac-8239-c6ee1f0101f1 service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:06:52.800510 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f726b88-dab0-42ac-8239-c6ee1f0101f1 service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:06:52.800999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f726b88-dab0-42ac-8239-c6ee1f0101f1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:06:52.814844 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f726b88-dab0-42ac-8239-c6ee1f0101f1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:06:52.817532 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f726b88-dab0-42ac-8239-c6ee1f0101f1 service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:06:52.829407 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1193/2358] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Aug 6 13:06:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 13:07:02.741993 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=38 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:07:02.824740 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=98 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:07:21.831118 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3404e9a-7099-41a9-9f09-b07be5821f87 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:21.831639 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3404e9a-7099-41a9-9f09-b07be5821f87 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:21.831639 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3404e9a-7099-41a9-9f09-b07be5821f87 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:21.833460 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1166/2359] 10.222.0.20 () {54 vars in 921 bytes} [Sat Aug 6 13:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 13:07:21.841537 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bbe9b-6cda-4af7-b833-6f96b3ebeabc None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:21.842016 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bbe9b-6cda-4af7-b833-6f96b3ebeabc None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:21.842085 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bbe9b-6cda-4af7-b833-6f96b3ebeabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:21.909814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-368bbe9b-6cda-4af7-b833-6f96b3ebeabc None None] MFA Rules not processed for user `524b728bf63c4ca88e0f25e531cf5056`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:07:21.913198 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-368bbe9b-6cda-4af7-b833-6f96b3ebeabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:07:21.934774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1194/2360] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Aug 6 13:07:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:07:21.945915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3404e9a-7099-41a9-9f09-b07be5821f87 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:21.947805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a3404e9a-7099-41a9-9f09-b07be5821f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:07:21.970825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3404e9a-7099-41a9-9f09-b07be5821f87 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:21.974799 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de902ce5-006e-4794-9c30-98e6a556fe0f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7610d6289efb44599d0cee03b93f9938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:21.975699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de902ce5-006e-4794-9c30-98e6a556fe0f service glance] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:21.976485 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de902ce5-006e-4794-9c30-98e6a556fe0f service glance] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:21.976952 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de902ce5-006e-4794-9c30-98e6a556fe0f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:21.989737 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de902ce5-006e-4794-9c30-98e6a556fe0f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:21.997507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de902ce5-006e-4794-9c30-98e6a556fe0f service glance] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:22.013346 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1167/2361] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:07:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:07:22.104647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc140f-0a2d-428c-8896-a7c68687fe01 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:22.105163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc140f-0a2d-428c-8896-a7c68687fe01 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:22.105504 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc140f-0a2d-428c-8896-a7c68687fe01 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:22.106965 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1195/2362] 10.222.0.20 () {54 vars in 921 bytes} [Sat Aug 6 13:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 13:07:22.111917 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24037ccc-6fb4-4865-a123-5900566b7f57 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:22.112352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24037ccc-6fb4-4865-a123-5900566b7f57 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:22.112702 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24037ccc-6fb4-4865-a123-5900566b7f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:22.152825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-24037ccc-6fb4-4865-a123-5900566b7f57 None None] MFA Rules not processed for user `55563aaa662246c5aaf107145475bd76`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:07:22.154145 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-24037ccc-6fb4-4865-a123-5900566b7f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:07:22.163316 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1168/2363] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Aug 6 13:07:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:07:22.175028 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc140f-0a2d-428c-8896-a7c68687fe01 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:22.186316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc140f-0a2d-428c-8896-a7c68687fe01 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:22.190111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc675fc0-8358-4639-8a0c-0a1ec78c5a84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:22.190709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc675fc0-8358-4639-8a0c-0a1ec78c5a84 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:22.190986 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc675fc0-8358-4639-8a0c-0a1ec78c5a84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:22.191253 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc675fc0-8358-4639-8a0c-0a1ec78c5a84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:22.198614 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc675fc0-8358-4639-8a0c-0a1ec78c5a84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:22.200111 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc675fc0-8358-4639-8a0c-0a1ec78c5a84 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:22.209255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1196/2364] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:07:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:07:22.294079 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179ed3c-7db1-427b-ba09-886298b6aa44 None None] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:22.294362 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179ed3c-7db1-427b-ba09-886298b6aa44 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:22.294658 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179ed3c-7db1-427b-ba09-886298b6aa44 None None] PATH_INFO: `/` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:22.296026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1169/2365] 10.222.0.20 () {54 vars in 921 bytes} [Sat Aug 6 13:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 06 13:07:22.301538 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ff6e7-8018-4fd0-b1ce-abcc2e94a0cd None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:22.301848 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ff6e7-8018-4fd0-b1ce-abcc2e94a0cd None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:22.302128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ff6e7-8018-4fd0-b1ce-abcc2e94a0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:22.322014 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1f6ff6e7-8018-4fd0-b1ce-abcc2e94a0cd None None] MFA Rules not processed for user `524b728bf63c4ca88e0f25e531cf5056`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:07:22.323213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1f6ff6e7-8018-4fd0-b1ce-abcc2e94a0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:07:22.333096 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1197/2366] 10.222.0.20 () {58 vars in 1018 bytes} [Sat Aug 6 13:07:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:07:22.343464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b179ed3c-7db1-427b-ba09-886298b6aa44 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:22.354816 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b179ed3c-7db1-427b-ba09-886298b6aa44 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:22.358416 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675896d-45e6-402f-b33f-44febe797285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:22.359015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675896d-45e6-402f-b33f-44febe797285 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:22.359296 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675896d-45e6-402f-b33f-44febe797285 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:22.359574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675896d-45e6-402f-b33f-44febe797285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:22.369571 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675896d-45e6-402f-b33f-44febe797285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:22.370991 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675896d-45e6-402f-b33f-44febe797285 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:22.381935 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1170/2367] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:07:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:07:28.581686 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b7188-23d8-4aed-9a92-a175303fa12b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:28.582200 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b7188-23d8-4aed-9a92-a175303fa12b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:28.582550 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b7188-23d8-4aed-9a92-a175303fa12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:28.610238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-dd9b7188-23d8-4aed-9a92-a175303fa12b None None] MFA Rules not processed for user `c810b52785b94c0cab4fd9f0d6e65825`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:07:28.611463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-dd9b7188-23d8-4aed-9a92-a175303fa12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:07:28.619549 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1198/2368] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Aug 6 13:07:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 13:07:28.631745 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675896d-45e6-402f-b33f-44febe797285 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:28.645439 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675896d-45e6-402f-b33f-44febe797285 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:28.650547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1e44b-e057-44f7-9457-5215831f81f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efc792a2655439f8b8070010059b6bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:28.651468 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1e44b-e057-44f7-9457-5215831f81f5 service swift] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:28.651748 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1e44b-e057-44f7-9457-5215831f81f5 service swift] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:28.652018 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1e44b-e057-44f7-9457-5215831f81f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:28.662403 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1e44b-e057-44f7-9457-5215831f81f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:28.664131 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1e44b-e057-44f7-9457-5215831f81f5 service swift] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:28.672254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1171/2369] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Aug 6 13:07:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 13:07:33.115763 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b7188-23d8-4aed-9a92-a175303fa12b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:33.126305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b7188-23d8-4aed-9a92-a175303fa12b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:33.136272 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfbe250-75f9-4a45-987d-5b7fdacb25e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:33.137739 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfbe250-75f9-4a45-987d-5b7fdacb25e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:33.138180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfbe250-75f9-4a45-987d-5b7fdacb25e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:33.138471 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfbe250-75f9-4a45-987d-5b7fdacb25e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:33.140457 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ebfbe250-75f9-4a45-987d-5b7fdacb25e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:07:33.231631 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfbe250-75f9-4a45-987d-5b7fdacb25e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:33.236457 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfbe250-75f9-4a45-987d-5b7fdacb25e6 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:33.247918 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1199/2370] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:07:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:07:37.161264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1e44b-e057-44f7-9457-5215831f81f5 service swift] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:37.183481 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1e44b-e057-44f7-9457-5215831f81f5 service swift] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:37.189338 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3d3f4-21fb-48a5-879f-2402faa64318 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2244655ba304b9c980b5e68772eefc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:37.190000 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3d3f4-21fb-48a5-879f-2402faa64318 service placement] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:37.190663 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3d3f4-21fb-48a5-879f-2402faa64318 service placement] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:37.191281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3d3f4-21fb-48a5-879f-2402faa64318 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:37.193947 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d2b3d3f4-21fb-48a5-879f-2402faa64318 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:07:37.208913 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3d3f4-21fb-48a5-879f-2402faa64318 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:37.210767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3d3f4-21fb-48a5-879f-2402faa64318 service placement] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:37.241659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1172/2371] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:07:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:07:37.283645 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfbe250-75f9-4a45-987d-5b7fdacb25e6 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:37.286793 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ebfbe250-75f9-4a45-987d-5b7fdacb25e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:07:37.303309 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfbe250-75f9-4a45-987d-5b7fdacb25e6 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:37.309109 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645ac86-0d34-456c-b45a-12015e9ccf45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:37.309960 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645ac86-0d34-456c-b45a-12015e9ccf45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:37.310385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645ac86-0d34-456c-b45a-12015e9ccf45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:37.310852 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645ac86-0d34-456c-b45a-12015e9ccf45 admin admin] PATH_INFO: `/v3/users/6844f315435f48f68488a2f0459bf155` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:37.312195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8645ac86-0d34-456c-b45a-12015e9ccf45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6844f315435f48f68488a2f0459bf155)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:37.314368 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8645ac86-0d34-456c-b45a-12015e9ccf45 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:37.393383 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8645ac86-0d34-456c-b45a-12015e9ccf45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6844f315435f48f68488a2f0459bf155'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:07:37.409592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8645ac86-0d34-456c-b45a-12015e9ccf45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6844f315435f48f68488a2f0459bf155'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:07:37.411041 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8645ac86-0d34-456c-b45a-12015e9ccf45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6844f315435f48f68488a2f0459bf155'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:07:37.411515 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8645ac86-0d34-456c-b45a-12015e9ccf45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6844f315435f48f68488a2f0459bf155'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:07:37.428354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8645ac86-0d34-456c-b45a-12015e9ccf45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6844f315435f48f68488a2f0459bf155'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:07:37.440268 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1200/2372] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:07:37 2022] DELETE /identity/v3/users/6844f315435f48f68488a2f0459bf155 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:07:37.632097 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3d3f4-21fb-48a5-879f-2402faa64318 service placement] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:37.643427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3d3f4-21fb-48a5-879f-2402faa64318 service placement] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:37.706640 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c6074-efa6-49b8-9b5d-9d79392f3957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:37.707279 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c6074-efa6-49b8-9b5d-9d79392f3957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:37.707557 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c6074-efa6-49b8-9b5d-9d79392f3957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:37.707832 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c6074-efa6-49b8-9b5d-9d79392f3957 admin admin] PATH_INFO: `/v3/projects/4931a52d9cda40b89d15804b69b5a2d7` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:37.709000 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c6074-efa6-49b8-9b5d-9d79392f3957 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4931a52d9cda40b89d15804b69b5a2d7)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:37.710896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c6074-efa6-49b8-9b5d-9d79392f3957 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:37.757808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3e6c6074-efa6-49b8-9b5d-9d79392f3957 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4931a52d9cda40b89d15804b69b5a2d7', 'request_id': 'req-3e6c6074-efa6-49b8-9b5d-9d79392f3957'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:07:37.764696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3e6c6074-efa6-49b8-9b5d-9d79392f3957 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4931a52d9cda40b89d15804b69b5a2d7', 'request_id': 'req-3e6c6074-efa6-49b8-9b5d-9d79392f3957'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:07:37.774015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3e6c6074-efa6-49b8-9b5d-9d79392f3957 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4931a52d9cda40b89d15804b69b5a2d7', 'request_id': 'req-3e6c6074-efa6-49b8-9b5d-9d79392f3957'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:07:37.786645 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3e6c6074-efa6-49b8-9b5d-9d79392f3957 admin admin] The token cache is being invalidate because project 4931a52d9cda40b89d15804b69b5a2d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:07:37.786926 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3e6c6074-efa6-49b8-9b5d-9d79392f3957 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:07:37.787547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3e6c6074-efa6-49b8-9b5d-9d79392f3957 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:07:37.789070 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1173/2373] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:07:37 2022] DELETE /identity/v3/projects/4931a52d9cda40b89d15804b69b5a2d7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:07:37.794056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa8a32-967b-49a3-abfa-e07ddc89ec40 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:37.794346 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa8a32-967b-49a3-abfa-e07ddc89ec40 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:37.794625 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa8a32-967b-49a3-abfa-e07ddc89ec40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:37.814513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-beaa8a32-967b-49a3-abfa-e07ddc89ec40 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:07:37.815721 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-beaa8a32-967b-49a3-abfa-e07ddc89ec40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:07:37.829407 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:07:37.877233 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1201/2374] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:07:37.942965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c83479-6e60-48d5-8f2f-0c6ca559437c None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:37.943240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c83479-6e60-48d5-8f2f-0c6ca559437c None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:37.943511 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c83479-6e60-48d5-8f2f-0c6ca559437c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:37.963309 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-35c83479-6e60-48d5-8f2f-0c6ca559437c None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:07:37.964544 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-35c83479-6e60-48d5-8f2f-0c6ca559437c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:07:37.972824 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1174/2375] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:07:37.977307 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa8a32-967b-49a3-abfa-e07ddc89ec40 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:37.986835 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa8a32-967b-49a3-abfa-e07ddc89ec40 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:37.991528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc8bc52-3c5d-4b29-87c0-c1e4eb20a2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:37.992140 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8bc52-3c5d-4b29-87c0-c1e4eb20a2a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:37.992449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8bc52-3c5d-4b29-87c0-c1e4eb20a2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:37.992724 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8bc52-3c5d-4b29-87c0-c1e4eb20a2a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:37.993421 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc8bc52-3c5d-4b29-87c0-c1e4eb20a2a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:07:37.993721 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc8bc52-3c5d-4b29-87c0-c1e4eb20a2a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:37.995345 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc8bc52-3c5d-4b29-87c0-c1e4eb20a2a9 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:38.007716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1202/2376] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 13:07:38.013041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c83479-6e60-48d5-8f2f-0c6ca559437c None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:38.025267 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c83479-6e60-48d5-8f2f-0c6ca559437c None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:38.030086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e69ab-9334-4b56-90e2-c2b7be3bf02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:38.030691 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e69ab-9334-4b56-90e2-c2b7be3bf02d admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:38.030967 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e69ab-9334-4b56-90e2-c2b7be3bf02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:38.031232 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e69ab-9334-4b56-90e2-c2b7be3bf02d admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:38.031917 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3e69ab-9334-4b56-90e2-c2b7be3bf02d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:38.033860 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3e69ab-9334-4b56-90e2-c2b7be3bf02d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:38.043168 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:07:38.064618 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1175/2377] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:07:38 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:07:38.069034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc8bc52-3c5d-4b29-87c0-c1e4eb20a2a9 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:38.080107 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc8bc52-3c5d-4b29-87c0-c1e4eb20a2a9 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:38.134349 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4b01c-1726-488f-9fc8-ad7dafebbb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:38.134965 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4b01c-1726-488f-9fc8-ad7dafebbb06 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:38.135245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4b01c-1726-488f-9fc8-ad7dafebbb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:38.135519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4b01c-1726-488f-9fc8-ad7dafebbb06 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:38.136178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4b01c-1726-488f-9fc8-ad7dafebbb06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:38.138838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4b01c-1726-488f-9fc8-ad7dafebbb06 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:38.159235 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1203/2378] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:07:38 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:07:38.163459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e69ab-9334-4b56-90e2-c2b7be3bf02d admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:38.175472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e69ab-9334-4b56-90e2-c2b7be3bf02d admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:38.180167 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13e1cd-2e6b-46a6-ae9d-0c06ac71d467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:38.180807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13e1cd-2e6b-46a6-ae9d-0c06ac71d467 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:38.181083 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13e1cd-2e6b-46a6-ae9d-0c06ac71d467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:38.181353 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13e1cd-2e6b-46a6-ae9d-0c06ac71d467 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:38.182041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13e1cd-2e6b-46a6-ae9d-0c06ac71d467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:38.184169 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13e1cd-2e6b-46a6-ae9d-0c06ac71d467 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:38.194065 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1176/2379] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:07:38.198153 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4b01c-1726-488f-9fc8-ad7dafebbb06 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:38.208679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4b01c-1726-488f-9fc8-ad7dafebbb06 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:38.213306 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9c911-5801-4798-89f0-d4a86d9eb93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:38.213908 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9c911-5801-4798-89f0-d4a86d9eb93c admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:38.214189 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9c911-5801-4798-89f0-d4a86d9eb93c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:38.214469 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9c911-5801-4798-89f0-d4a86d9eb93c admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:38.215194 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9c911-5801-4798-89f0-d4a86d9eb93c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:38.216825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9c911-5801-4798-89f0-d4a86d9eb93c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:38.227595 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1204/2380] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 13:07:38.231934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13e1cd-2e6b-46a6-ae9d-0c06ac71d467 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:38.241583 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13e1cd-2e6b-46a6-ae9d-0c06ac71d467 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:38.246832 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e953db7-084c-458c-bfa6-1cee590aa05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:38.247475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e953db7-084c-458c-bfa6-1cee590aa05e admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:38.247754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e953db7-084c-458c-bfa6-1cee590aa05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:38.248029 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e953db7-084c-458c-bfa6-1cee590aa05e admin admin] PATH_INFO: `/v3/projects/91ea5c47859b4e7aa1005b018c9219e0/users/296d95d15c2f49038175269113919dc3/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:38.262159 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e953db7-084c-458c-bfa6-1cee590aa05e admin admin] RBAC: Authorizing `identity:create_grant(project_id=91ea5c47859b4e7aa1005b018c9219e0, user_id=296d95d15c2f49038175269113919dc3, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:38.264790 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e953db7-084c-458c-bfa6-1cee590aa05e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:38.277801 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1177/2381] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:07:38 2022] PUT /identity/v3/projects/91ea5c47859b4e7aa1005b018c9219e0/users/296d95d15c2f49038175269113919dc3/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:07:38.287664 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0663c9-349b-4719-ac0e-513e21acf8cb None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:38.288145 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0663c9-349b-4719-ac0e-513e21acf8cb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:38.288245 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0663c9-349b-4719-ac0e-513e21acf8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:38.316850 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9a0663c9-349b-4719-ac0e-513e21acf8cb None None] MFA Rules not processed for user `296d95d15c2f49038175269113919dc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:07:38.318267 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9a0663c9-349b-4719-ac0e-513e21acf8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:07:38.382353 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1205/2382] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:07:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:07:38.386864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e953db7-084c-458c-bfa6-1cee590aa05e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:38.445710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e953db7-084c-458c-bfa6-1cee590aa05e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:38.450417 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb7b45-642a-4dac-9ca5-8719379f7d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:38.451025 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb7b45-642a-4dac-9ca5-8719379f7d26 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:38.451300 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb7b45-642a-4dac-9ca5-8719379f7d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:38.451568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb7b45-642a-4dac-9ca5-8719379f7d26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:38.452358 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb7b45-642a-4dac-9ca5-8719379f7d26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:38.454133 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb7b45-642a-4dac-9ca5-8719379f7d26 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:38.483063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1178/2383] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:07:38 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:07:38.487161 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0663c9-349b-4719-ac0e-513e21acf8cb None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:38.498361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0663c9-349b-4719-ac0e-513e21acf8cb None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:38.509718 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:07:38.556172 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42be29-a9f0-4f79-92e2-e0d3644d18d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:38.556840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42be29-a9f0-4f79-92e2-e0d3644d18d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:38.557124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42be29-a9f0-4f79-92e2-e0d3644d18d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:38.557400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42be29-a9f0-4f79-92e2-e0d3644d18d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:38.558070 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42be29-a9f0-4f79-92e2-e0d3644d18d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:38.560520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42be29-a9f0-4f79-92e2-e0d3644d18d6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:38.581715 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1206/2384] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:07:38 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:07:38.585797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb7b45-642a-4dac-9ca5-8719379f7d26 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:38.596836 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb7b45-642a-4dac-9ca5-8719379f7d26 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:38.601955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd6693-d5bc-46a6-b294-83a182ad010b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:38.602640 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd6693-d5bc-46a6-b294-83a182ad010b admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:38.602981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd6693-d5bc-46a6-b294-83a182ad010b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:38.603355 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd6693-d5bc-46a6-b294-83a182ad010b admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:38.604108 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dd6693-d5bc-46a6-b294-83a182ad010b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:38.606454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dd6693-d5bc-46a6-b294-83a182ad010b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:38.616901 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1179/2385] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:07:38.621038 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42be29-a9f0-4f79-92e2-e0d3644d18d6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:38.631106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42be29-a9f0-4f79-92e2-e0d3644d18d6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:38.635801 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccdd0da-a19d-4752-8a67-cf7052827175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:38.636498 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdd0da-a19d-4752-8a67-cf7052827175 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:38.636806 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdd0da-a19d-4752-8a67-cf7052827175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:38.637086 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccdd0da-a19d-4752-8a67-cf7052827175 admin admin] PATH_INFO: `/v3/projects/6ede27d6c0de4072ac37f1246f89d8df/users/6c1da91dcd5d4d56a4bd15942b948965/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:38.651447 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccdd0da-a19d-4752-8a67-cf7052827175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ede27d6c0de4072ac37f1246f89d8df, user_id=6c1da91dcd5d4d56a4bd15942b948965, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:38.654666 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccdd0da-a19d-4752-8a67-cf7052827175 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:38.672988 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1207/2386] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:07:38 2022] PUT /identity/v3/projects/6ede27d6c0de4072ac37f1246f89d8df/users/6c1da91dcd5d4d56a4bd15942b948965/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:07:38.676974 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd6693-d5bc-46a6-b294-83a182ad010b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:38.686911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd6693-d5bc-46a6-b294-83a182ad010b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:38.741374 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1419ee3-437a-4bb7-af16-1da221da2da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:38.741977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1419ee3-437a-4bb7-af16-1da221da2da9 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:38.742258 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1419ee3-437a-4bb7-af16-1da221da2da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:38.742531 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1419ee3-437a-4bb7-af16-1da221da2da9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:38.743213 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1419ee3-437a-4bb7-af16-1da221da2da9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:38.744842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1419ee3-437a-4bb7-af16-1da221da2da9 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:38.754251 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1180/2387] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:07:38.758301 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccdd0da-a19d-4752-8a67-cf7052827175 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:38.768393 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccdd0da-a19d-4752-8a67-cf7052827175 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:38.772987 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1277f14-f787-4816-87ca-ef47e24ba9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:38.773633 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1277f14-f787-4816-87ca-ef47e24ba9e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:38.773908 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1277f14-f787-4816-87ca-ef47e24ba9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:38.774179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1277f14-f787-4816-87ca-ef47e24ba9e7 admin admin] PATH_INFO: `/v3/projects/6ede27d6c0de4072ac37f1246f89d8df/users/6c1da91dcd5d4d56a4bd15942b948965/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:38.776078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1277f14-f787-4816-87ca-ef47e24ba9e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ede27d6c0de4072ac37f1246f89d8df, user_id=6c1da91dcd5d4d56a4bd15942b948965, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:38.779233 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1277f14-f787-4816-87ca-ef47e24ba9e7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:38.789380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1208/2388] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:07:38 2022] PUT /identity/v3/projects/6ede27d6c0de4072ac37f1246f89d8df/users/6c1da91dcd5d4d56a4bd15942b948965/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:07:38.799576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ffb49-67c1-4f42-982a-d3fba6bdd1ba None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:38.799861 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ffb49-67c1-4f42-982a-d3fba6bdd1ba None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:38.800135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ffb49-67c1-4f42-982a-d3fba6bdd1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:38.827048 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-802ffb49-67c1-4f42-982a-d3fba6bdd1ba None None] MFA Rules not processed for user `6c1da91dcd5d4d56a4bd15942b948965`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:07:38.828237 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-802ffb49-67c1-4f42-982a-d3fba6bdd1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:07:38.840194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:07:38.895947 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1181/2389] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:07:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:07:38.908345 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1277f14-f787-4816-87ca-ef47e24ba9e7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:38.910071 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d1277f14-f787-4816-87ca-ef47e24ba9e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:07:38.977625 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1277f14-f787-4816-87ca-ef47e24ba9e7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:38.981769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb867fa-edb2-43e4-8401-485d75e82bbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:38.982388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb867fa-edb2-43e4-8401-485d75e82bbc service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:38.982668 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb867fa-edb2-43e4-8401-485d75e82bbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:38.982941 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb867fa-edb2-43e4-8401-485d75e82bbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:38.998469 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb867fa-edb2-43e4-8401-485d75e82bbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:38.999941 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb867fa-edb2-43e4-8401-485d75e82bbc service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:39.009803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1209/2390] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:07:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:07:39.084066 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ffb49-67c1-4f42-982a-d3fba6bdd1ba None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:39.102449 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ffb49-67c1-4f42-982a-d3fba6bdd1ba None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:39.106379 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0265f-9ea6-456a-b327-0a94a8ca10d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:39.106990 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0265f-9ea6-456a-b327-0a94a8ca10d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:39.107266 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0265f-9ea6-456a-b327-0a94a8ca10d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:39.107544 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0265f-9ea6-456a-b327-0a94a8ca10d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:39.162472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf0265f-9ea6-456a-b327-0a94a8ca10d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:39.163899 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf0265f-9ea6-456a-b327-0a94a8ca10d0 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:39.173047 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1182/2391] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:07:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:07:42.502761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb867fa-edb2-43e4-8401-485d75e82bbc service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:42.504385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0bb867fa-edb2-43e4-8401-485d75e82bbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:07:42.573219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb867fa-edb2-43e4-8401-485d75e82bbc service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:42.577716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc78c37-c308-4aca-90f6-b0e22d3f09a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:42.578334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc78c37-c308-4aca-90f6-b0e22d3f09a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:42.578611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc78c37-c308-4aca-90f6-b0e22d3f09a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:42.578880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc78c37-c308-4aca-90f6-b0e22d3f09a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:42.580707 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3cc78c37-c308-4aca-90f6-b0e22d3f09a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:07:42.591158 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc78c37-c308-4aca-90f6-b0e22d3f09a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:42.592599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc78c37-c308-4aca-90f6-b0e22d3f09a7 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:42.600935 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1210/2392] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:07:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:07:49.096803 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0265f-9ea6-456a-b327-0a94a8ca10d0 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:49.106793 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0265f-9ea6-456a-b327-0a94a8ca10d0 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:49.110845 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7128c-78d8-4efb-80e0-98f5f1aeb321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:49.111466 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7128c-78d8-4efb-80e0-98f5f1aeb321 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:49.111746 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7128c-78d8-4efb-80e0-98f5f1aeb321 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:49.112014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7128c-78d8-4efb-80e0-98f5f1aeb321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:49.114125 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a0e7128c-78d8-4efb-80e0-98f5f1aeb321 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:07:49.162463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc78c37-c308-4aca-90f6-b0e22d3f09a7 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:49.167391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:07:49.173316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc78c37-c308-4aca-90f6-b0e22d3f09a7 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:49.178062 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08432081-5433-4251-a0c5-53e04218feba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:49.178720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08432081-5433-4251-a0c5-53e04218feba service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:49.179002 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08432081-5433-4251-a0c5-53e04218feba service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:49.179284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08432081-5433-4251-a0c5-53e04218feba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:49.181812 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-08432081-5433-4251-a0c5-53e04218feba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:07:49.237506 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7128c-78d8-4efb-80e0-98f5f1aeb321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:49.239134 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7128c-78d8-4efb-80e0-98f5f1aeb321 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:49.251912 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1183/2393] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:07:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:07:49.277890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08432081-5433-4251-a0c5-53e04218feba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:49.279329 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08432081-5433-4251-a0c5-53e04218feba service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:49.291563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1211/2394] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:07:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:07:52.880159 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7128c-78d8-4efb-80e0-98f5f1aeb321 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:52.881533 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a0e7128c-78d8-4efb-80e0-98f5f1aeb321 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:07:52.895876 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7128c-78d8-4efb-80e0-98f5f1aeb321 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:52.901482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3e383-2364-47dd-af98-0691d790936f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:52.902255 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3e383-2364-47dd-af98-0691d790936f service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:52.902503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3e383-2364-47dd-af98-0691d790936f service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:52.902882 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3e383-2364-47dd-af98-0691d790936f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:52.905113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-03d3e383-2364-47dd-af98-0691d790936f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:07:52.924756 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:07:52.964804 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d3e383-2364-47dd-af98-0691d790936f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:52.966413 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d3e383-2364-47dd-af98-0691d790936f service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:52.975800 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1184/2395] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:07:52 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:07:55.524233 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1168e-2d9a-4442-aeca-37a4c0c5a885 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:55.524731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1168e-2d9a-4442-aeca-37a4c0c5a885 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:55.524814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1168e-2d9a-4442-aeca-37a4c0c5a885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:55.544139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3cb1168e-2d9a-4442-aeca-37a4c0c5a885 None None] MFA Rules not processed for user `c810b52785b94c0cab4fd9f0d6e65825`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:07:55.545372 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3cb1168e-2d9a-4442-aeca-37a4c0c5a885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:07:55.603997 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1212/2396] 10.222.0.20 () {58 vars in 967 bytes} [Sat Aug 6 13:07:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 13:07:55.626690 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3e383-2364-47dd-af98-0691d790936f service nova] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:55.628271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-03d3e383-2364-47dd-af98-0691d790936f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:07:55.710974 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3e383-2364-47dd-af98-0691d790936f service nova] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:55.716833 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39553b5-4a14-4522-a229-b8ce4763cde8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efc792a2655439f8b8070010059b6bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:55.717787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39553b5-4a14-4522-a229-b8ce4763cde8 service swift] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:55.718063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39553b5-4a14-4522-a229-b8ce4763cde8 service swift] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:55.718383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39553b5-4a14-4522-a229-b8ce4763cde8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:55.733520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39553b5-4a14-4522-a229-b8ce4763cde8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:55.734980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39553b5-4a14-4522-a229-b8ce4763cde8 service swift] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:55.743492 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1185/2397] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Aug 6 13:07:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 13:07:58.226536 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1168e-2d9a-4442-aeca-37a4c0c5a885 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:07:58.228038 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3cb1168e-2d9a-4442-aeca-37a4c0c5a885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:07:58.242742 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1168e-2d9a-4442-aeca-37a4c0c5a885 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:07:58.253461 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba171389-e904-44fc-a067-95304407e84f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:07:58.254096 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba171389-e904-44fc-a067-95304407e84f service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:07:58.254376 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba171389-e904-44fc-a067-95304407e84f service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:07:58.254653 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba171389-e904-44fc-a067-95304407e84f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:07:58.257970 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ba171389-e904-44fc-a067-95304407e84f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:07:58.323507 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba171389-e904-44fc-a067-95304407e84f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:07:58.325051 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba171389-e904-44fc-a067-95304407e84f service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:07:58.334913 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1213/2398] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:07:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:08:05.740027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=50 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:08.330433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=88 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:27.120465 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39553b5-4a14-4522-a229-b8ce4763cde8 service swift] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:27.179557 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39553b5-4a14-4522-a229-b8ce4763cde8 service swift] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:27.183975 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec953a-5e54-466f-afbf-7253f57f8c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:27.184631 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec953a-5e54-466f-afbf-7253f57f8c10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:27.184907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec953a-5e54-466f-afbf-7253f57f8c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:27.185175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec953a-5e54-466f-afbf-7253f57f8c10 admin admin] PATH_INFO: `/v3/users/296d95d15c2f49038175269113919dc3` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:27.186842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec953a-5e54-466f-afbf-7253f57f8c10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=296d95d15c2f49038175269113919dc3)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:27.188782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec953a-5e54-466f-afbf-7253f57f8c10 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:27.233613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-41ec953a-5e54-466f-afbf-7253f57f8c10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '296d95d15c2f49038175269113919dc3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:27.240468 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-41ec953a-5e54-466f-afbf-7253f57f8c10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '296d95d15c2f49038175269113919dc3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:27.252520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-41ec953a-5e54-466f-afbf-7253f57f8c10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '296d95d15c2f49038175269113919dc3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:27.253175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-41ec953a-5e54-466f-afbf-7253f57f8c10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '296d95d15c2f49038175269113919dc3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:27.253777 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-41ec953a-5e54-466f-afbf-7253f57f8c10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '296d95d15c2f49038175269113919dc3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:27.267443 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1186/2399] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:08:27 2022] DELETE /identity/v3/users/296d95d15c2f49038175269113919dc3 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:27.271916 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42479a-aff9-4c45-9d3d-58796f8e514a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:27.272427 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42479a-aff9-4c45-9d3d-58796f8e514a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:27.272427 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42479a-aff9-4c45-9d3d-58796f8e514a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:27.290963 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8d42479a-aff9-4c45-9d3d-58796f8e514a None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:08:27.292149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8d42479a-aff9-4c45-9d3d-58796f8e514a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:27.349837 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1214/2400] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:08:27.359921 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec953a-5e54-466f-afbf-7253f57f8c10 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:27.361362 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-41ec953a-5e54-466f-afbf-7253f57f8c10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:27.417503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec953a-5e54-466f-afbf-7253f57f8c10 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:27.421194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f76e5-5178-4e17-80a5-72867d6e961b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:27.421804 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f76e5-5178-4e17-80a5-72867d6e961b service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:27.422084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f76e5-5178-4e17-80a5-72867d6e961b service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:27.422357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f76e5-5178-4e17-80a5-72867d6e961b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:27.431206 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f76e5-5178-4e17-80a5-72867d6e961b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:27.432737 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f76e5-5178-4e17-80a5-72867d6e961b service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:27.442523 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1187/2401] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:08:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:08:27.719811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42479a-aff9-4c45-9d3d-58796f8e514a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:27.721265 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8d42479a-aff9-4c45-9d3d-58796f8e514a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:27.736355 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42479a-aff9-4c45-9d3d-58796f8e514a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:27.740989 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3c881-9810-4ab2-98c3-da376bbe82b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:27.741640 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3c881-9810-4ab2-98c3-da376bbe82b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:27.742559 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3c881-9810-4ab2-98c3-da376bbe82b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:27.743012 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3c881-9810-4ab2-98c3-da376bbe82b8 admin admin] PATH_INFO: `/v3/projects/91ea5c47859b4e7aa1005b018c9219e0` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:27.744200 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd3c881-9810-4ab2-98c3-da376bbe82b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91ea5c47859b4e7aa1005b018c9219e0)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:27.746272 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd3c881-9810-4ab2-98c3-da376bbe82b8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:27.792336 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ebd3c881-9810-4ab2-98c3-da376bbe82b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91ea5c47859b4e7aa1005b018c9219e0', 'request_id': 'req-ebd3c881-9810-4ab2-98c3-da376bbe82b8'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:27.798140 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ebd3c881-9810-4ab2-98c3-da376bbe82b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91ea5c47859b4e7aa1005b018c9219e0', 'request_id': 'req-ebd3c881-9810-4ab2-98c3-da376bbe82b8'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:27.805281 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ebd3c881-9810-4ab2-98c3-da376bbe82b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91ea5c47859b4e7aa1005b018c9219e0', 'request_id': 'req-ebd3c881-9810-4ab2-98c3-da376bbe82b8'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:27.816753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ebd3c881-9810-4ab2-98c3-da376bbe82b8 admin admin] The token cache is being invalidate because project 91ea5c47859b4e7aa1005b018c9219e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:08:27.817038 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ebd3c881-9810-4ab2-98c3-da376bbe82b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:27.817654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ebd3c881-9810-4ab2-98c3-da376bbe82b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:27.819136 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1215/2402] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:08:27 2022] DELETE /identity/v3/projects/91ea5c47859b4e7aa1005b018c9219e0 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:27.823085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f76e5-5178-4e17-80a5-72867d6e961b service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:27.832878 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f76e5-5178-4e17-80a5-72867d6e961b service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:27.850694 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:27.889356 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebe7a4-3a8c-4b07-9920-551c7a7cd5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:27.889990 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebe7a4-3a8c-4b07-9920-551c7a7cd5a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:27.890269 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebe7a4-3a8c-4b07-9920-551c7a7cd5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:27.890542 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebe7a4-3a8c-4b07-9920-551c7a7cd5a2 admin admin] PATH_INFO: `/v3/users/6c1da91dcd5d4d56a4bd15942b948965` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:27.891682 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebe7a4-3a8c-4b07-9920-551c7a7cd5a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c1da91dcd5d4d56a4bd15942b948965)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:27.893650 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebe7a4-3a8c-4b07-9920-551c7a7cd5a2 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:27.938389 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-23ebe7a4-3a8c-4b07-9920-551c7a7cd5a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c1da91dcd5d4d56a4bd15942b948965'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:27.945188 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-23ebe7a4-3a8c-4b07-9920-551c7a7cd5a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c1da91dcd5d4d56a4bd15942b948965'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:27.956983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-23ebe7a4-3a8c-4b07-9920-551c7a7cd5a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c1da91dcd5d4d56a4bd15942b948965'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:27.957631 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-23ebe7a4-3a8c-4b07-9920-551c7a7cd5a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c1da91dcd5d4d56a4bd15942b948965'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:27.958250 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-23ebe7a4-3a8c-4b07-9920-551c7a7cd5a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c1da91dcd5d4d56a4bd15942b948965'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:27.971335 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1188/2403] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:08:27 2022] DELETE /identity/v3/users/6c1da91dcd5d4d56a4bd15942b948965 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:08:28.245414 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3c881-9810-4ab2-98c3-da376bbe82b8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:28.246972 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ebd3c881-9810-4ab2-98c3-da376bbe82b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:28.262572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3c881-9810-4ab2-98c3-da376bbe82b8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:28.325452 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a7bf5-9cb0-40ae-9619-f818c7f04687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:28.326077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a7bf5-9cb0-40ae-9619-f818c7f04687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:28.326301 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a7bf5-9cb0-40ae-9619-f818c7f04687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:28.326573 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a7bf5-9cb0-40ae-9619-f818c7f04687 admin admin] PATH_INFO: `/v3/projects/6ede27d6c0de4072ac37f1246f89d8df` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:28.328416 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a7bf5-9cb0-40ae-9619-f818c7f04687 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ede27d6c0de4072ac37f1246f89d8df)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:28.330343 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a7bf5-9cb0-40ae-9619-f818c7f04687 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:28.364267 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:28.381843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-153a7bf5-9cb0-40ae-9619-f818c7f04687 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ede27d6c0de4072ac37f1246f89d8df', 'request_id': 'req-153a7bf5-9cb0-40ae-9619-f818c7f04687'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:28.387306 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-153a7bf5-9cb0-40ae-9619-f818c7f04687 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ede27d6c0de4072ac37f1246f89d8df', 'request_id': 'req-153a7bf5-9cb0-40ae-9619-f818c7f04687'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:28.395094 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-153a7bf5-9cb0-40ae-9619-f818c7f04687 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ede27d6c0de4072ac37f1246f89d8df', 'request_id': 'req-153a7bf5-9cb0-40ae-9619-f818c7f04687'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:28.407101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-153a7bf5-9cb0-40ae-9619-f818c7f04687 admin admin] The token cache is being invalidate because project 6ede27d6c0de4072ac37f1246f89d8df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:08:28.407392 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-153a7bf5-9cb0-40ae-9619-f818c7f04687 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:28.408018 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-153a7bf5-9cb0-40ae-9619-f818c7f04687 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:28.409575 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1216/2404] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:08:28 2022] DELETE /identity/v3/projects/6ede27d6c0de4072ac37f1246f89d8df => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:28.414449 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba07b3-e376-4c0e-b11f-6556a1ec0609 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:28.414911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba07b3-e376-4c0e-b11f-6556a1ec0609 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:28.415023 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba07b3-e376-4c0e-b11f-6556a1ec0609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:28.434077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-08ba07b3-e376-4c0e-b11f-6556a1ec0609 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:08:28.435280 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-08ba07b3-e376-4c0e-b11f-6556a1ec0609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:28.491725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1189/2405] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:08:28.499282 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b26e02-f04b-4cfe-b97b-29bc2944bfb5 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:28.499563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b26e02-f04b-4cfe-b97b-29bc2944bfb5 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:28.499837 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b26e02-f04b-4cfe-b97b-29bc2944bfb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:28.519813 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d9b26e02-f04b-4cfe-b97b-29bc2944bfb5 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:08:28.521067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d9b26e02-f04b-4cfe-b97b-29bc2944bfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:28.529114 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1217/2406] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:08:28.533914 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba07b3-e376-4c0e-b11f-6556a1ec0609 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:28.543349 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba07b3-e376-4c0e-b11f-6556a1ec0609 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:28.548500 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88129b90-0329-4717-83b6-da5c7c242bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:28.548684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88129b90-0329-4717-83b6-da5c7c242bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:28.548962 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88129b90-0329-4717-83b6-da5c7c242bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:28.549230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88129b90-0329-4717-83b6-da5c7c242bd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:28.549928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88129b90-0329-4717-83b6-da5c7c242bd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:08:28.550225 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88129b90-0329-4717-83b6-da5c7c242bd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:28.551820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88129b90-0329-4717-83b6-da5c7c242bd4 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:28.563647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1190/2407] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 13:08:28.568015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b26e02-f04b-4cfe-b97b-29bc2944bfb5 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:28.577930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b26e02-f04b-4cfe-b97b-29bc2944bfb5 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:28.582491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c162223-1a34-4b3c-809d-fbdb72b38a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:28.583096 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c162223-1a34-4b3c-809d-fbdb72b38a18 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:28.583373 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c162223-1a34-4b3c-809d-fbdb72b38a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:28.583643 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c162223-1a34-4b3c-809d-fbdb72b38a18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:28.584331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c162223-1a34-4b3c-809d-fbdb72b38a18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:28.586247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c162223-1a34-4b3c-809d-fbdb72b38a18 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:28.614257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1218/2408] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:08:28 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:08:28.618270 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88129b90-0329-4717-83b6-da5c7c242bd4 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:28.628292 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88129b90-0329-4717-83b6-da5c7c242bd4 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:28.656641 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:28.689223 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c640b-4052-4908-bc57-e809e2eb89f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:28.689838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c640b-4052-4908-bc57-e809e2eb89f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:28.690120 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c640b-4052-4908-bc57-e809e2eb89f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:28.690396 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c640b-4052-4908-bc57-e809e2eb89f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:28.691058 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c640b-4052-4908-bc57-e809e2eb89f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:28.693034 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c640b-4052-4908-bc57-e809e2eb89f5 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:28.713820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1191/2409] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:08:28 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:08:28.717977 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c162223-1a34-4b3c-809d-fbdb72b38a18 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:28.728632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c162223-1a34-4b3c-809d-fbdb72b38a18 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:28.733552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70ae4c-9f2d-4108-95f9-f16bdebc8f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:28.734159 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70ae4c-9f2d-4108-95f9-f16bdebc8f8f admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:28.734444 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70ae4c-9f2d-4108-95f9-f16bdebc8f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:28.734721 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70ae4c-9f2d-4108-95f9-f16bdebc8f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:28.735413 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b70ae4c-9f2d-4108-95f9-f16bdebc8f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:28.737067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b70ae4c-9f2d-4108-95f9-f16bdebc8f8f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:28.747234 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1219/2410] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:08:28.751765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c640b-4052-4908-bc57-e809e2eb89f5 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:28.761552 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c640b-4052-4908-bc57-e809e2eb89f5 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:28.766708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1895167e-07c8-4bd2-860a-8495805f656f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:28.767305 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895167e-07c8-4bd2-860a-8495805f656f admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:28.767581 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895167e-07c8-4bd2-860a-8495805f656f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:28.767853 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895167e-07c8-4bd2-860a-8495805f656f admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:28.768569 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1895167e-07c8-4bd2-860a-8495805f656f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:28.770143 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1895167e-07c8-4bd2-860a-8495805f656f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:28.780611 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1192/2411] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:08:28.784757 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70ae4c-9f2d-4108-95f9-f16bdebc8f8f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:28.794600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70ae4c-9f2d-4108-95f9-f16bdebc8f8f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:28.799416 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ef019-818d-42ce-9c4d-20bf955e8da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:28.800070 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ef019-818d-42ce-9c4d-20bf955e8da9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:28.800378 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ef019-818d-42ce-9c4d-20bf955e8da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:28.800657 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ef019-818d-42ce-9c4d-20bf955e8da9 admin admin] PATH_INFO: `/v3/projects/f0ec00aaa57e4c03b58127975a24a947/users/1031b24610a044f883b9fdcba4048d05/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:28.815726 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ef019-818d-42ce-9c4d-20bf955e8da9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0ec00aaa57e4c03b58127975a24a947, user_id=1031b24610a044f883b9fdcba4048d05, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:28.818432 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ef019-818d-42ce-9c4d-20bf955e8da9 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:28.832620 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1220/2412] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:08:28 2022] PUT /identity/v3/projects/f0ec00aaa57e4c03b58127975a24a947/users/1031b24610a044f883b9fdcba4048d05/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:28.843350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4537d-4abf-4f2e-ad9f-e51e0f860051 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:28.843634 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4537d-4abf-4f2e-ad9f-e51e0f860051 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:28.843905 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4537d-4abf-4f2e-ad9f-e51e0f860051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:28.870761 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-62c4537d-4abf-4f2e-ad9f-e51e0f860051 None None] MFA Rules not processed for user `1031b24610a044f883b9fdcba4048d05`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:08:28.871953 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-62c4537d-4abf-4f2e-ad9f-e51e0f860051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:28.934807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1193/2413] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:08:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:08:28.939935 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ef019-818d-42ce-9c4d-20bf955e8da9 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:29.003219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ef019-818d-42ce-9c4d-20bf955e8da9 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:29.007835 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9e3c7-c4bd-4e3b-946c-c23f2611e6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:29.008488 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9e3c7-c4bd-4e3b-946c-c23f2611e6ea admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:29.008771 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9e3c7-c4bd-4e3b-946c-c23f2611e6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:29.009045 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9e3c7-c4bd-4e3b-946c-c23f2611e6ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:29.009716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9e3c7-c4bd-4e3b-946c-c23f2611e6ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:29.011613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab9e3c7-c4bd-4e3b-946c-c23f2611e6ea admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:29.038290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1221/2414] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:08:28 2022] POST /identity/v3/projects => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:08:29.042283 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4537d-4abf-4f2e-ad9f-e51e0f860051 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:29.052976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4537d-4abf-4f2e-ad9f-e51e0f860051 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:29.107192 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3cf7fb-441b-4ae4-a5c5-943c5c98a48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:29.107732 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3cf7fb-441b-4ae4-a5c5-943c5c98a48a admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:29.108556 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3cf7fb-441b-4ae4-a5c5-943c5c98a48a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:29.108831 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3cf7fb-441b-4ae4-a5c5-943c5c98a48a admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:29.109480 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3cf7fb-441b-4ae4-a5c5-943c5c98a48a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:29.111386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3cf7fb-441b-4ae4-a5c5-943c5c98a48a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:29.132481 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1194/2415] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:08:29 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:08:29.136374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9e3c7-c4bd-4e3b-946c-c23f2611e6ea admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:29.146415 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9e3c7-c4bd-4e3b-946c-c23f2611e6ea admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:29.151354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b1694-4e4d-4a1e-a01c-ba78d3dd4c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:29.151963 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b1694-4e4d-4a1e-a01c-ba78d3dd4c34 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:29.152265 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b1694-4e4d-4a1e-a01c-ba78d3dd4c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:29.152586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b1694-4e4d-4a1e-a01c-ba78d3dd4c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:29.153282 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b1694-4e4d-4a1e-a01c-ba78d3dd4c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:29.154870 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b1694-4e4d-4a1e-a01c-ba78d3dd4c34 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:29.165378 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1222/2416] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:08:29.169403 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3cf7fb-441b-4ae4-a5c5-943c5c98a48a admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:29.179765 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3cf7fb-441b-4ae4-a5c5-943c5c98a48a admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:29.184469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc247f-6d78-476b-b825-71461b25c9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:29.185141 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc247f-6d78-476b-b825-71461b25c9b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:29.185422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc247f-6d78-476b-b825-71461b25c9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:29.185730 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc247f-6d78-476b-b825-71461b25c9b7 admin admin] PATH_INFO: `/v3/projects/0b2a4160d0ae43c3b190cbbe9f394e1f/users/cdf24e621e7345ddace479154a704d36/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:29.200818 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc247f-6d78-476b-b825-71461b25c9b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b2a4160d0ae43c3b190cbbe9f394e1f, user_id=cdf24e621e7345ddace479154a704d36, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:29.203500 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc247f-6d78-476b-b825-71461b25c9b7 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:29.215760 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1195/2417] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:08:29 2022] PUT /identity/v3/projects/0b2a4160d0ae43c3b190cbbe9f394e1f/users/cdf24e621e7345ddace479154a704d36/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:29.219533 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b1694-4e4d-4a1e-a01c-ba78d3dd4c34 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:29.230939 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b1694-4e4d-4a1e-a01c-ba78d3dd4c34 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:29.251648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:29.288834 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390950da-0504-415d-97bb-32d59b02c96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:29.289437 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390950da-0504-415d-97bb-32d59b02c96c admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:29.289725 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390950da-0504-415d-97bb-32d59b02c96c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:29.290002 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390950da-0504-415d-97bb-32d59b02c96c admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:29.290690 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390950da-0504-415d-97bb-32d59b02c96c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:29.292275 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390950da-0504-415d-97bb-32d59b02c96c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:29.302465 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1223/2418] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:08:29.306584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc247f-6d78-476b-b825-71461b25c9b7 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:29.312625 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:29.319240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc247f-6d78-476b-b825-71461b25c9b7 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:29.324029 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e04b00-8f11-4ad4-8025-77af1cd43767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:29.324736 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e04b00-8f11-4ad4-8025-77af1cd43767 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:29.325018 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e04b00-8f11-4ad4-8025-77af1cd43767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:29.325297 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e04b00-8f11-4ad4-8025-77af1cd43767 admin admin] PATH_INFO: `/v3/projects/0b2a4160d0ae43c3b190cbbe9f394e1f/users/cdf24e621e7345ddace479154a704d36/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:29.327314 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e04b00-8f11-4ad4-8025-77af1cd43767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b2a4160d0ae43c3b190cbbe9f394e1f, user_id=cdf24e621e7345ddace479154a704d36, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:29.329987 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e04b00-8f11-4ad4-8025-77af1cd43767 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:29.340373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1196/2419] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:08:29 2022] PUT /identity/v3/projects/0b2a4160d0ae43c3b190cbbe9f394e1f/users/cdf24e621e7345ddace479154a704d36/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:29.350504 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd7e00-3aea-4d75-8f9c-7ad4ac57d2da None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:29.350989 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd7e00-3aea-4d75-8f9c-7ad4ac57d2da None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:29.351178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd7e00-3aea-4d75-8f9c-7ad4ac57d2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:29.379060 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3ecd7e00-3aea-4d75-8f9c-7ad4ac57d2da None None] MFA Rules not processed for user `cdf24e621e7345ddace479154a704d36`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:08:29.380197 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3ecd7e00-3aea-4d75-8f9c-7ad4ac57d2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:29.445815 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1224/2420] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:08:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 13:08:29.456396 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e04b00-8f11-4ad4-8025-77af1cd43767 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:29.458103 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d7e04b00-8f11-4ad4-8025-77af1cd43767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:29.513398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e04b00-8f11-4ad4-8025-77af1cd43767 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:29.517193 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f046e-c676-4b34-b6de-373e48997337 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:29.517784 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f046e-c676-4b34-b6de-373e48997337 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:29.518059 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f046e-c676-4b34-b6de-373e48997337 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:29.518329 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f046e-c676-4b34-b6de-373e48997337 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:29.527325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f046e-c676-4b34-b6de-373e48997337 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:29.528820 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f046e-c676-4b34-b6de-373e48997337 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:29.538638 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1197/2421] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:08:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:08:30.866473 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd7e00-3aea-4d75-8f9c-7ad4ac57d2da None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:30.928044 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd7e00-3aea-4d75-8f9c-7ad4ac57d2da None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:30.932649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2172d7-6dee-4283-9415-cc14274ab2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:30.933302 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2172d7-6dee-4283-9415-cc14274ab2f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:30.933592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2172d7-6dee-4283-9415-cc14274ab2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:30.933864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2172d7-6dee-4283-9415-cc14274ab2f8 admin admin] PATH_INFO: `/v3/users/1031b24610a044f883b9fdcba4048d05` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:30.935039 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2172d7-6dee-4283-9415-cc14274ab2f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1031b24610a044f883b9fdcba4048d05)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:30.937718 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2172d7-6dee-4283-9415-cc14274ab2f8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:30.990818 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8e2172d7-6dee-4283-9415-cc14274ab2f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1031b24610a044f883b9fdcba4048d05'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:30.991978 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:31.004895 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8e2172d7-6dee-4283-9415-cc14274ab2f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1031b24610a044f883b9fdcba4048d05'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:31.005617 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8e2172d7-6dee-4283-9415-cc14274ab2f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1031b24610a044f883b9fdcba4048d05'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:31.006237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8e2172d7-6dee-4283-9415-cc14274ab2f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1031b24610a044f883b9fdcba4048d05'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:31.013406 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8e2172d7-6dee-4283-9415-cc14274ab2f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1031b24610a044f883b9fdcba4048d05'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:31.025630 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1225/2422] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:08:30 2022] DELETE /identity/v3/users/1031b24610a044f883b9fdcba4048d05 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:31.030752 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824e917-f6b7-4adb-8ae3-50a7002d6986 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:31.031192 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824e917-f6b7-4adb-8ae3-50a7002d6986 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:31.031278 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824e917-f6b7-4adb-8ae3-50a7002d6986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:31.050573 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-2824e917-f6b7-4adb-8ae3-50a7002d6986 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:08:31.051750 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2824e917-f6b7-4adb-8ae3-50a7002d6986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:31.107657 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1198/2423] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 13:08:31.117707 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2172d7-6dee-4283-9415-cc14274ab2f8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:31.119124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8e2172d7-6dee-4283-9415-cc14274ab2f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:31.174837 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2172d7-6dee-4283-9415-cc14274ab2f8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:31.178519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51160c3-6467-41de-ae67-3ebc300541e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:31.179044 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51160c3-6467-41de-ae67-3ebc300541e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:31.179324 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51160c3-6467-41de-ae67-3ebc300541e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:31.179595 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51160c3-6467-41de-ae67-3ebc300541e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:31.188293 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51160c3-6467-41de-ae67-3ebc300541e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:31.189722 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51160c3-6467-41de-ae67-3ebc300541e2 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:31.198899 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1226/2424] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:08:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:08:31.469619 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2824e917-f6b7-4adb-8ae3-50a7002d6986 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:31.471041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2824e917-f6b7-4adb-8ae3-50a7002d6986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:31.482909 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2824e917-f6b7-4adb-8ae3-50a7002d6986 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:31.487376 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3031d-5779-4800-8ee8-47926ea1f416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:31.488012 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3031d-5779-4800-8ee8-47926ea1f416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:31.488294 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3031d-5779-4800-8ee8-47926ea1f416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:31.488595 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3031d-5779-4800-8ee8-47926ea1f416 admin admin] PATH_INFO: `/v3/projects/f0ec00aaa57e4c03b58127975a24a947` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:31.489771 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3031d-5779-4800-8ee8-47926ea1f416 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0ec00aaa57e4c03b58127975a24a947)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:31.491678 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3031d-5779-4800-8ee8-47926ea1f416 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:31.540345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a0f3031d-5779-4800-8ee8-47926ea1f416 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0ec00aaa57e4c03b58127975a24a947', 'request_id': 'req-a0f3031d-5779-4800-8ee8-47926ea1f416'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:31.545948 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a0f3031d-5779-4800-8ee8-47926ea1f416 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0ec00aaa57e4c03b58127975a24a947', 'request_id': 'req-a0f3031d-5779-4800-8ee8-47926ea1f416'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:31.553931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a0f3031d-5779-4800-8ee8-47926ea1f416 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0ec00aaa57e4c03b58127975a24a947', 'request_id': 'req-a0f3031d-5779-4800-8ee8-47926ea1f416'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:31.565148 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a0f3031d-5779-4800-8ee8-47926ea1f416 admin admin] The token cache is being invalidate because project f0ec00aaa57e4c03b58127975a24a947 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:08:31.565427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a0f3031d-5779-4800-8ee8-47926ea1f416 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:31.566075 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a0f3031d-5779-4800-8ee8-47926ea1f416 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:31.567616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1199/2425] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:08:31 2022] DELETE /identity/v3/projects/f0ec00aaa57e4c03b58127975a24a947 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:31.571618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51160c3-6467-41de-ae67-3ebc300541e2 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:31.584146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51160c3-6467-41de-ae67-3ebc300541e2 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:31.640746 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ba513-ac37-433c-b280-8a1b82c9de50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:31.641378 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ba513-ac37-433c-b280-8a1b82c9de50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:31.641657 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ba513-ac37-433c-b280-8a1b82c9de50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:31.641930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ba513-ac37-433c-b280-8a1b82c9de50 admin admin] PATH_INFO: `/v3/users/cdf24e621e7345ddace479154a704d36` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:31.643070 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ba513-ac37-433c-b280-8a1b82c9de50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdf24e621e7345ddace479154a704d36)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:31.645008 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ba513-ac37-433c-b280-8a1b82c9de50 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:31.697985 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d91ba513-ac37-433c-b280-8a1b82c9de50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdf24e621e7345ddace479154a704d36'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:31.711093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d91ba513-ac37-433c-b280-8a1b82c9de50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdf24e621e7345ddace479154a704d36'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:31.711818 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d91ba513-ac37-433c-b280-8a1b82c9de50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdf24e621e7345ddace479154a704d36'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:31.712406 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d91ba513-ac37-433c-b280-8a1b82c9de50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdf24e621e7345ddace479154a704d36'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:31.716020 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:31.720766 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d91ba513-ac37-433c-b280-8a1b82c9de50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdf24e621e7345ddace479154a704d36'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:31.734936 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1227/2426] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:08:31 2022] DELETE /identity/v3/users/cdf24e621e7345ddace479154a704d36 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:32.002258 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3031d-5779-4800-8ee8-47926ea1f416 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:32.003705 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a0f3031d-5779-4800-8ee8-47926ea1f416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:32.017073 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3031d-5779-4800-8ee8-47926ea1f416 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:32.029039 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:32.081477 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47feb0-ba47-4294-88e7-cf9811ab383f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:32.082106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47feb0-ba47-4294-88e7-cf9811ab383f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:32.082390 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47feb0-ba47-4294-88e7-cf9811ab383f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:32.082665 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47feb0-ba47-4294-88e7-cf9811ab383f admin admin] PATH_INFO: `/v3/projects/0b2a4160d0ae43c3b190cbbe9f394e1f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:32.083827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47feb0-ba47-4294-88e7-cf9811ab383f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b2a4160d0ae43c3b190cbbe9f394e1f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:32.085802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47feb0-ba47-4294-88e7-cf9811ab383f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:32.132628 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fb47feb0-ba47-4294-88e7-cf9811ab383f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b2a4160d0ae43c3b190cbbe9f394e1f', 'request_id': 'req-fb47feb0-ba47-4294-88e7-cf9811ab383f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:32.138113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fb47feb0-ba47-4294-88e7-cf9811ab383f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b2a4160d0ae43c3b190cbbe9f394e1f', 'request_id': 'req-fb47feb0-ba47-4294-88e7-cf9811ab383f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:32.144690 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fb47feb0-ba47-4294-88e7-cf9811ab383f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b2a4160d0ae43c3b190cbbe9f394e1f', 'request_id': 'req-fb47feb0-ba47-4294-88e7-cf9811ab383f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:32.155816 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fb47feb0-ba47-4294-88e7-cf9811ab383f admin admin] The token cache is being invalidate because project 0b2a4160d0ae43c3b190cbbe9f394e1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:08:32.156100 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fb47feb0-ba47-4294-88e7-cf9811ab383f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:32.156739 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fb47feb0-ba47-4294-88e7-cf9811ab383f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:32.158228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1200/2427] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:08:32 2022] DELETE /identity/v3/projects/0b2a4160d0ae43c3b190cbbe9f394e1f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:32.163092 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d0496-8abe-4a8c-bf9c-912b6fab93bf None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:32.163567 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d0496-8abe-4a8c-bf9c-912b6fab93bf None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:32.163654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d0496-8abe-4a8c-bf9c-912b6fab93bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:32.184538 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c05d0496-8abe-4a8c-bf9c-912b6fab93bf None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:08:32.185731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c05d0496-8abe-4a8c-bf9c-912b6fab93bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:32.243025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1228/2428] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:08:32.251243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4e133-707c-4823-8fb8-7d5a889ab5c7 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:32.251588 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4e133-707c-4823-8fb8-7d5a889ab5c7 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:32.251951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4e133-707c-4823-8fb8-7d5a889ab5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:32.271661 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-5da4e133-707c-4823-8fb8-7d5a889ab5c7 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:08:32.272978 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5da4e133-707c-4823-8fb8-7d5a889ab5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:32.280823 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1201/2429] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:08:32.285023 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d0496-8abe-4a8c-bf9c-912b6fab93bf None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:32.294501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d0496-8abe-4a8c-bf9c-912b6fab93bf None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:32.299470 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a8c4e-413e-4159-bc0e-d8f4c3fd29c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:32.300064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a8c4e-413e-4159-bc0e-d8f4c3fd29c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:32.300441 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a8c4e-413e-4159-bc0e-d8f4c3fd29c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:32.300714 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a8c4e-413e-4159-bc0e-d8f4c3fd29c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:32.301426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a8c4e-413e-4159-bc0e-d8f4c3fd29c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:08:32.301724 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a8c4e-413e-4159-bc0e-d8f4c3fd29c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:32.303291 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3a8c4e-413e-4159-bc0e-d8f4c3fd29c5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:32.315118 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1229/2430] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 13:08:32.319246 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4e133-707c-4823-8fb8-7d5a889ab5c7 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:32.328563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4e133-707c-4823-8fb8-7d5a889ab5c7 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:32.333196 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd2622-5686-4afa-8ddd-3ed744911ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:32.333791 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd2622-5686-4afa-8ddd-3ed744911ebb admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:32.334069 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd2622-5686-4afa-8ddd-3ed744911ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:32.334339 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd2622-5686-4afa-8ddd-3ed744911ebb admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:32.334981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd2622-5686-4afa-8ddd-3ed744911ebb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:32.336911 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd2622-5686-4afa-8ddd-3ed744911ebb admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:32.363437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1202/2431] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:08:32 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:08:32.367584 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a8c4e-413e-4159-bc0e-d8f4c3fd29c5 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:32.379240 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a8c4e-413e-4159-bc0e-d8f4c3fd29c5 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:32.433039 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad0c16-b3cb-4d4e-a63d-93f2723829bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:32.433641 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad0c16-b3cb-4d4e-a63d-93f2723829bf admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:32.433918 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad0c16-b3cb-4d4e-a63d-93f2723829bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:32.434188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad0c16-b3cb-4d4e-a63d-93f2723829bf admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:32.434845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad0c16-b3cb-4d4e-a63d-93f2723829bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:32.437382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad0c16-b3cb-4d4e-a63d-93f2723829bf admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:32.457638 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1230/2432] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:08:32 2022] POST /identity/v3/users => generated 285 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:08:32.461756 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd2622-5686-4afa-8ddd-3ed744911ebb admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:32.472376 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd2622-5686-4afa-8ddd-3ed744911ebb admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:32.477216 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e85a6-483e-4d68-843c-818ffd4b754b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:32.477812 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e85a6-483e-4d68-843c-818ffd4b754b admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:32.478091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e85a6-483e-4d68-843c-818ffd4b754b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:32.478364 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e85a6-483e-4d68-843c-818ffd4b754b admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:32.479048 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e85a6-483e-4d68-843c-818ffd4b754b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:32.480673 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e85a6-483e-4d68-843c-818ffd4b754b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:32.490920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1203/2433] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:08:32.494504 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad0c16-b3cb-4d4e-a63d-93f2723829bf admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:32.504264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad0c16-b3cb-4d4e-a63d-93f2723829bf admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:32.508756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb641c63-e301-44b6-89a5-195b4b9d7a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:32.509345 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb641c63-e301-44b6-89a5-195b4b9d7a40 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:32.509625 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb641c63-e301-44b6-89a5-195b4b9d7a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:32.509899 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb641c63-e301-44b6-89a5-195b4b9d7a40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:32.510581 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb641c63-e301-44b6-89a5-195b4b9d7a40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:32.512162 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb641c63-e301-44b6-89a5-195b4b9d7a40 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:32.522535 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1231/2434] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:08:32.527440 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e85a6-483e-4d68-843c-818ffd4b754b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:32.538598 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e85a6-483e-4d68-843c-818ffd4b754b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:32.543139 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b067a6-2cf6-411e-b572-0fdb46415a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:32.543789 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b067a6-2cf6-411e-b572-0fdb46415a2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:32.544066 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b067a6-2cf6-411e-b572-0fdb46415a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:32.544358 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b067a6-2cf6-411e-b572-0fdb46415a2b admin admin] PATH_INFO: `/v3/projects/69bcd2e4cbca48cdb5b78b7a0b6a8c90/users/af2667de58da4c2c809049368504bc9a/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:32.559293 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b067a6-2cf6-411e-b572-0fdb46415a2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=69bcd2e4cbca48cdb5b78b7a0b6a8c90, user_id=af2667de58da4c2c809049368504bc9a, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:32.561983 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b067a6-2cf6-411e-b572-0fdb46415a2b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:32.574993 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1204/2435] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:08:32 2022] PUT /identity/v3/projects/69bcd2e4cbca48cdb5b78b7a0b6a8c90/users/af2667de58da4c2c809049368504bc9a/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:32.586324 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf6b55-a860-4d14-97aa-358a41e59069 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:32.586610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf6b55-a860-4d14-97aa-358a41e59069 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:32.586887 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf6b55-a860-4d14-97aa-358a41e59069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:32.614701 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6ebf6b55-a860-4d14-97aa-358a41e59069 None None] MFA Rules not processed for user `af2667de58da4c2c809049368504bc9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:08:32.615891 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6ebf6b55-a860-4d14-97aa-358a41e59069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:32.657608 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:32.681352 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1232/2436] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:08:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:08:32.685777 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b067a6-2cf6-411e-b572-0fdb46415a2b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:32.729539 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:32.749974 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b067a6-2cf6-411e-b572-0fdb46415a2b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:32.754750 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa13ac2-02bf-4f64-b4bc-96fed9859110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:32.755306 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa13ac2-02bf-4f64-b4bc-96fed9859110 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:32.755587 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa13ac2-02bf-4f64-b4bc-96fed9859110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:32.755858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa13ac2-02bf-4f64-b4bc-96fed9859110 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:32.756558 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa13ac2-02bf-4f64-b4bc-96fed9859110 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:32.758448 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa13ac2-02bf-4f64-b4bc-96fed9859110 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:32.785876 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1205/2437] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:08:32 2022] POST /identity/v3/projects => generated 369 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:08:32.789790 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf6b55-a860-4d14-97aa-358a41e59069 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:32.800953 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf6b55-a860-4d14-97aa-358a41e59069 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:32.856702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1091a2-6ead-4df1-ac94-07f22c4a60ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:32.857306 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1091a2-6ead-4df1-ac94-07f22c4a60ed admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:32.857583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1091a2-6ead-4df1-ac94-07f22c4a60ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:32.857851 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1091a2-6ead-4df1-ac94-07f22c4a60ed admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:32.858508 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1091a2-6ead-4df1-ac94-07f22c4a60ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:32.860428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1091a2-6ead-4df1-ac94-07f22c4a60ed admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:32.880618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1233/2438] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:08:32 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:08:32.884497 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa13ac2-02bf-4f64-b4bc-96fed9859110 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:32.894907 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa13ac2-02bf-4f64-b4bc-96fed9859110 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:32.899570 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac643cad-ed94-4243-a0ca-a5fb0dc28fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:32.900171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac643cad-ed94-4243-a0ca-a5fb0dc28fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:32.900507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac643cad-ed94-4243-a0ca-a5fb0dc28fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:32.900779 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac643cad-ed94-4243-a0ca-a5fb0dc28fb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:32.901468 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac643cad-ed94-4243-a0ca-a5fb0dc28fb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:32.903042 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac643cad-ed94-4243-a0ca-a5fb0dc28fb7 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:32.914281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1206/2439] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:08:32.918104 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1091a2-6ead-4df1-ac94-07f22c4a60ed admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:32.927710 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1091a2-6ead-4df1-ac94-07f22c4a60ed admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:32.932580 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebf199-5694-4300-89aa-a06baf3250c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:32.933236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebf199-5694-4300-89aa-a06baf3250c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:32.933519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebf199-5694-4300-89aa-a06baf3250c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:32.933797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebf199-5694-4300-89aa-a06baf3250c6 admin admin] PATH_INFO: `/v3/projects/25b031a37c93427b86503443ab067822/users/077f44a37ca74aa78d1fcbab532a361e/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:32.948240 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaebf199-5694-4300-89aa-a06baf3250c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25b031a37c93427b86503443ab067822, user_id=077f44a37ca74aa78d1fcbab532a361e, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:32.950956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaebf199-5694-4300-89aa-a06baf3250c6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:32.963854 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1234/2440] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:08:32 2022] PUT /identity/v3/projects/25b031a37c93427b86503443ab067822/users/077f44a37ca74aa78d1fcbab532a361e/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:32.968239 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac643cad-ed94-4243-a0ca-a5fb0dc28fb7 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:32.978242 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac643cad-ed94-4243-a0ca-a5fb0dc28fb7 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:33.033206 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa9040-3feb-4416-bc78-660f5e46b7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:33.033816 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa9040-3feb-4416-bc78-660f5e46b7c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:33.034095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa9040-3feb-4416-bc78-660f5e46b7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:33.034372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa9040-3feb-4416-bc78-660f5e46b7c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:33.035052 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa9040-3feb-4416-bc78-660f5e46b7c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:33.036681 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aa9040-3feb-4416-bc78-660f5e46b7c1 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:33.046278 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1207/2441] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 13:08:33.051522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebf199-5694-4300-89aa-a06baf3250c6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:33.063051 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebf199-5694-4300-89aa-a06baf3250c6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:33.067660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742ae3b-c5a5-4296-8ac6-8e32832c784f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:33.068326 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742ae3b-c5a5-4296-8ac6-8e32832c784f admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:33.068721 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742ae3b-c5a5-4296-8ac6-8e32832c784f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:33.068997 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742ae3b-c5a5-4296-8ac6-8e32832c784f admin admin] PATH_INFO: `/v3/projects/25b031a37c93427b86503443ab067822/users/077f44a37ca74aa78d1fcbab532a361e/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:33.070911 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742ae3b-c5a5-4296-8ac6-8e32832c784f admin admin] RBAC: Authorizing `identity:create_grant(project_id=25b031a37c93427b86503443ab067822, user_id=077f44a37ca74aa78d1fcbab532a361e, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:33.073559 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742ae3b-c5a5-4296-8ac6-8e32832c784f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:33.084130 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1235/2442] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:08:33 2022] PUT /identity/v3/projects/25b031a37c93427b86503443ab067822/users/077f44a37ca74aa78d1fcbab532a361e/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:33.094689 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93bfb8-f011-4386-8eef-5be3e35eaf7e None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:33.095132 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93bfb8-f011-4386-8eef-5be3e35eaf7e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:33.095262 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93bfb8-f011-4386-8eef-5be3e35eaf7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:33.123867 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ec93bfb8-f011-4386-8eef-5be3e35eaf7e None None] MFA Rules not processed for user `077f44a37ca74aa78d1fcbab532a361e`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:08:33.125123 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ec93bfb8-f011-4386-8eef-5be3e35eaf7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:33.191155 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1208/2443] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:08:33 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 13:08:33.202814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742ae3b-c5a5-4296-8ac6-8e32832c784f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:33.204538 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f742ae3b-c5a5-4296-8ac6-8e32832c784f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:33.260249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742ae3b-c5a5-4296-8ac6-8e32832c784f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:33.264583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af757cdd-dd93-4a4c-ad9a-2478e330aa2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:33.265192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af757cdd-dd93-4a4c-ad9a-2478e330aa2a service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:33.265471 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af757cdd-dd93-4a4c-ad9a-2478e330aa2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:33.265747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af757cdd-dd93-4a4c-ad9a-2478e330aa2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:33.278192 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af757cdd-dd93-4a4c-ad9a-2478e330aa2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:33.279617 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af757cdd-dd93-4a4c-ad9a-2478e330aa2a service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:33.289200 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1236/2444] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:08:33 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:08:33.333105 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec93bfb8-f011-4386-8eef-5be3e35eaf7e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:33.334482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ec93bfb8-f011-4386-8eef-5be3e35eaf7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:33.346474 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec93bfb8-f011-4386-8eef-5be3e35eaf7e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:33.350282 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200e4d1-eac3-4017-a924-d05abd3859ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:33.350886 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200e4d1-eac3-4017-a924-d05abd3859ea service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:33.351162 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200e4d1-eac3-4017-a924-d05abd3859ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:33.351431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200e4d1-eac3-4017-a924-d05abd3859ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:33.383621 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:33.407078 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6200e4d1-eac3-4017-a924-d05abd3859ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:33.408579 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6200e4d1-eac3-4017-a924-d05abd3859ea service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:33.417260 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1209/2445] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:08:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:08:36.675445 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af757cdd-dd93-4a4c-ad9a-2478e330aa2a service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:36.715826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:36.738400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af757cdd-dd93-4a4c-ad9a-2478e330aa2a service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:36.742938 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd235cb7-e990-4e23-98ac-6de01b1b9b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:36.743567 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd235cb7-e990-4e23-98ac-6de01b1b9b00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:36.743846 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd235cb7-e990-4e23-98ac-6de01b1b9b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:36.744117 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd235cb7-e990-4e23-98ac-6de01b1b9b00 admin admin] PATH_INFO: `/v3/users/af2667de58da4c2c809049368504bc9a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:36.745287 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd235cb7-e990-4e23-98ac-6de01b1b9b00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af2667de58da4c2c809049368504bc9a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:36.747326 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd235cb7-e990-4e23-98ac-6de01b1b9b00 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:36.791695 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-fd235cb7-e990-4e23-98ac-6de01b1b9b00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af2667de58da4c2c809049368504bc9a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:36.803732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-fd235cb7-e990-4e23-98ac-6de01b1b9b00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af2667de58da4c2c809049368504bc9a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:36.804412 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-fd235cb7-e990-4e23-98ac-6de01b1b9b00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af2667de58da4c2c809049368504bc9a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:36.805024 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-fd235cb7-e990-4e23-98ac-6de01b1b9b00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af2667de58da4c2c809049368504bc9a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:36.813180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-fd235cb7-e990-4e23-98ac-6de01b1b9b00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af2667de58da4c2c809049368504bc9a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:36.825064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1237/2446] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:08:36 2022] DELETE /identity/v3/users/af2667de58da4c2c809049368504bc9a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:36.829561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afeeda7-42d6-4409-a060-651c1b9d89ed None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:36.830019 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afeeda7-42d6-4409-a060-651c1b9d89ed None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:36.830140 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afeeda7-42d6-4409-a060-651c1b9d89ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:36.848349 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-7afeeda7-42d6-4409-a060-651c1b9d89ed None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:08:36.849541 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7afeeda7-42d6-4409-a060-651c1b9d89ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:36.907784 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1210/2447] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:08:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:08:36.917283 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd235cb7-e990-4e23-98ac-6de01b1b9b00 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:36.918700 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fd235cb7-e990-4e23-98ac-6de01b1b9b00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:36.983539 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd235cb7-e990-4e23-98ac-6de01b1b9b00 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:36.987206 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7942bf6-3c41-4896-b5ea-c4bc7d39b967 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:36.987828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7942bf6-3c41-4896-b5ea-c4bc7d39b967 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:36.988110 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7942bf6-3c41-4896-b5ea-c4bc7d39b967 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:36.988423 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7942bf6-3c41-4896-b5ea-c4bc7d39b967 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:37.000362 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7942bf6-3c41-4896-b5ea-c4bc7d39b967 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:37.001793 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7942bf6-3c41-4896-b5ea-c4bc7d39b967 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:37.011432 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1238/2448] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:08:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:08:37.281507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afeeda7-42d6-4409-a060-651c1b9d89ed None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:37.283067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7afeeda7-42d6-4409-a060-651c1b9d89ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:37.296431 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afeeda7-42d6-4409-a060-651c1b9d89ed None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:37.301124 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcb8f0-b779-4d0f-8265-bfb5d4cced8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:37.301758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcb8f0-b779-4d0f-8265-bfb5d4cced8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:37.302039 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcb8f0-b779-4d0f-8265-bfb5d4cced8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:37.302315 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcb8f0-b779-4d0f-8265-bfb5d4cced8d admin admin] PATH_INFO: `/v3/projects/69bcd2e4cbca48cdb5b78b7a0b6a8c90` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:37.303471 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bcb8f0-b779-4d0f-8265-bfb5d4cced8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=69bcd2e4cbca48cdb5b78b7a0b6a8c90)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:37.305422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bcb8f0-b779-4d0f-8265-bfb5d4cced8d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:37.352813 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-71bcb8f0-b779-4d0f-8265-bfb5d4cced8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69bcd2e4cbca48cdb5b78b7a0b6a8c90', 'request_id': 'req-71bcb8f0-b779-4d0f-8265-bfb5d4cced8d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:37.358272 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-71bcb8f0-b779-4d0f-8265-bfb5d4cced8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69bcd2e4cbca48cdb5b78b7a0b6a8c90', 'request_id': 'req-71bcb8f0-b779-4d0f-8265-bfb5d4cced8d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:37.364941 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-71bcb8f0-b779-4d0f-8265-bfb5d4cced8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69bcd2e4cbca48cdb5b78b7a0b6a8c90', 'request_id': 'req-71bcb8f0-b779-4d0f-8265-bfb5d4cced8d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:37.376010 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-71bcb8f0-b779-4d0f-8265-bfb5d4cced8d admin admin] The token cache is being invalidate because project 69bcd2e4cbca48cdb5b78b7a0b6a8c90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:08:37.376316 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-71bcb8f0-b779-4d0f-8265-bfb5d4cced8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:37.376955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-71bcb8f0-b779-4d0f-8265-bfb5d4cced8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:37.378469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1211/2449] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:08:37 2022] DELETE /identity/v3/projects/69bcd2e4cbca48cdb5b78b7a0b6a8c90 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:08:37.382588 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7942bf6-3c41-4896-b5ea-c4bc7d39b967 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:37.392613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7942bf6-3c41-4896-b5ea-c4bc7d39b967 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:37.452442 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e3ec7b-746e-4088-aeae-5019a548c009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:37.453005 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3ec7b-746e-4088-aeae-5019a548c009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:37.453286 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3ec7b-746e-4088-aeae-5019a548c009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:37.453559 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3ec7b-746e-4088-aeae-5019a548c009 admin admin] PATH_INFO: `/v3/users/077f44a37ca74aa78d1fcbab532a361e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:37.454727 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e3ec7b-746e-4088-aeae-5019a548c009 admin admin] RBAC: Authorizing `identity:delete_user(user_id=077f44a37ca74aa78d1fcbab532a361e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:37.456714 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e3ec7b-746e-4088-aeae-5019a548c009 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:37.458327 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:37.503573 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-75e3ec7b-746e-4088-aeae-5019a548c009 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '077f44a37ca74aa78d1fcbab532a361e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:37.515578 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-75e3ec7b-746e-4088-aeae-5019a548c009 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '077f44a37ca74aa78d1fcbab532a361e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:37.516228 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-75e3ec7b-746e-4088-aeae-5019a548c009 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '077f44a37ca74aa78d1fcbab532a361e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:37.516858 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-75e3ec7b-746e-4088-aeae-5019a548c009 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '077f44a37ca74aa78d1fcbab532a361e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:37.523730 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-75e3ec7b-746e-4088-aeae-5019a548c009 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '077f44a37ca74aa78d1fcbab532a361e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:37.536244 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1239/2450] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:08:37 2022] DELETE /identity/v3/users/077f44a37ca74aa78d1fcbab532a361e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:37.799362 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcb8f0-b779-4d0f-8265-bfb5d4cced8d admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:37.800883 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-71bcb8f0-b779-4d0f-8265-bfb5d4cced8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:37.815928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcb8f0-b779-4d0f-8265-bfb5d4cced8d admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:37.871965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77779aa3-eb68-405e-8e23-4ea9f2b8af9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:37.872696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77779aa3-eb68-405e-8e23-4ea9f2b8af9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:37.872974 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77779aa3-eb68-405e-8e23-4ea9f2b8af9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:37.873733 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77779aa3-eb68-405e-8e23-4ea9f2b8af9d admin admin] PATH_INFO: `/v3/projects/25b031a37c93427b86503443ab067822` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:37.874864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77779aa3-eb68-405e-8e23-4ea9f2b8af9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=25b031a37c93427b86503443ab067822)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:37.876811 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77779aa3-eb68-405e-8e23-4ea9f2b8af9d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:37.878257 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:37.923393 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-77779aa3-eb68-405e-8e23-4ea9f2b8af9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25b031a37c93427b86503443ab067822', 'request_id': 'req-77779aa3-eb68-405e-8e23-4ea9f2b8af9d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:37.928683 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-77779aa3-eb68-405e-8e23-4ea9f2b8af9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25b031a37c93427b86503443ab067822', 'request_id': 'req-77779aa3-eb68-405e-8e23-4ea9f2b8af9d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:37.935425 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-77779aa3-eb68-405e-8e23-4ea9f2b8af9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25b031a37c93427b86503443ab067822', 'request_id': 'req-77779aa3-eb68-405e-8e23-4ea9f2b8af9d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:37.946626 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-77779aa3-eb68-405e-8e23-4ea9f2b8af9d admin admin] The token cache is being invalidate because project 25b031a37c93427b86503443ab067822 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:08:37.946909 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-77779aa3-eb68-405e-8e23-4ea9f2b8af9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:37.947531 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-77779aa3-eb68-405e-8e23-4ea9f2b8af9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:37.949056 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1212/2451] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:08:37 2022] DELETE /identity/v3/projects/25b031a37c93427b86503443ab067822 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:37.954568 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee01afa-848a-41e5-a8b8-22d4f80abc23 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:37.955050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee01afa-848a-41e5-a8b8-22d4f80abc23 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:37.955156 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee01afa-848a-41e5-a8b8-22d4f80abc23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:37.974379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6ee01afa-848a-41e5-a8b8-22d4f80abc23 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:08:37.975754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6ee01afa-848a-41e5-a8b8-22d4f80abc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:38.038057 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1240/2452] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:08:38.047364 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc719c7-89ae-43ee-b3a8-27891a2ebff5 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:38.047705 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc719c7-89ae-43ee-b3a8-27891a2ebff5 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:38.047984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc719c7-89ae-43ee-b3a8-27891a2ebff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:38.072611 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-fbc719c7-89ae-43ee-b3a8-27891a2ebff5 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:08:38.073823 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fbc719c7-89ae-43ee-b3a8-27891a2ebff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:38.082469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1213/2453] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:08:38.086808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee01afa-848a-41e5-a8b8-22d4f80abc23 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:38.096644 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee01afa-848a-41e5-a8b8-22d4f80abc23 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:38.101443 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7dee28-ba8b-46d0-8c71-18541134da87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:38.102129 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7dee28-ba8b-46d0-8c71-18541134da87 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:38.102410 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7dee28-ba8b-46d0-8c71-18541134da87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:38.102683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7dee28-ba8b-46d0-8c71-18541134da87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:38.103371 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7dee28-ba8b-46d0-8c71-18541134da87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:08:38.103669 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7dee28-ba8b-46d0-8c71-18541134da87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:38.105267 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7dee28-ba8b-46d0-8c71-18541134da87 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:38.116921 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1241/2454] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 13:08:38.120942 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc719c7-89ae-43ee-b3a8-27891a2ebff5 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:38.130578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc719c7-89ae-43ee-b3a8-27891a2ebff5 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:38.135320 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5268f-bf70-4a90-81c4-78c9f2e08f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:38.135920 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5268f-bf70-4a90-81c4-78c9f2e08f8f admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:38.136194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5268f-bf70-4a90-81c4-78c9f2e08f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:38.136507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5268f-bf70-4a90-81c4-78c9f2e08f8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:38.137169 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d5268f-bf70-4a90-81c4-78c9f2e08f8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:38.139123 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d5268f-bf70-4a90-81c4-78c9f2e08f8f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:38.165989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1214/2455] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:08:38 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:08:38.170343 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7dee28-ba8b-46d0-8c71-18541134da87 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:38.180599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7dee28-ba8b-46d0-8c71-18541134da87 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:38.236744 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4573d9-70f8-4855-8f4d-f4d47acf7b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:38.237280 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4573d9-70f8-4855-8f4d-f4d47acf7b97 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:38.237561 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4573d9-70f8-4855-8f4d-f4d47acf7b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:38.237833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4573d9-70f8-4855-8f4d-f4d47acf7b97 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:38.238483 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4573d9-70f8-4855-8f4d-f4d47acf7b97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:38.240410 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4573d9-70f8-4855-8f4d-f4d47acf7b97 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:38.251709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:38.263866 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1242/2456] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:08:38 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:08:38.267898 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5268f-bf70-4a90-81c4-78c9f2e08f8f admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:38.278071 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5268f-bf70-4a90-81c4-78c9f2e08f8f admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:38.282641 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137cb745-6736-418d-9c09-41952b1fe5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:38.283235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cb745-6736-418d-9c09-41952b1fe5d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:38.283514 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cb745-6736-418d-9c09-41952b1fe5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:38.283786 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cb745-6736-418d-9c09-41952b1fe5d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:38.284508 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137cb745-6736-418d-9c09-41952b1fe5d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:38.286110 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137cb745-6736-418d-9c09-41952b1fe5d8 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:38.296169 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1215/2457] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:08:38.299900 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4573d9-70f8-4855-8f4d-f4d47acf7b97 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:38.309503 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4573d9-70f8-4855-8f4d-f4d47acf7b97 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:38.314681 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2e887-d1d0-48fc-9d5c-1461237438ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:38.315268 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2e887-d1d0-48fc-9d5c-1461237438ac admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:38.315545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2e887-d1d0-48fc-9d5c-1461237438ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:38.315815 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2e887-d1d0-48fc-9d5c-1461237438ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:38.316557 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa2e887-d1d0-48fc-9d5c-1461237438ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:38.318127 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa2e887-d1d0-48fc-9d5c-1461237438ac admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:38.328448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1243/2458] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:08:38.332242 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137cb745-6736-418d-9c09-41952b1fe5d8 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:38.341995 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137cb745-6736-418d-9c09-41952b1fe5d8 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:38.346551 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0ae5a-6da5-4016-b48f-cb0d8d9766e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:38.347188 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0ae5a-6da5-4016-b48f-cb0d8d9766e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:38.347464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0ae5a-6da5-4016-b48f-cb0d8d9766e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:38.347733 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0ae5a-6da5-4016-b48f-cb0d8d9766e8 admin admin] PATH_INFO: `/v3/projects/c3914bffa8e24e51aabe2b7b58ff65b1/users/9438c91ac19c49b4a711d4d9aafdeac5/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:38.362861 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e0ae5a-6da5-4016-b48f-cb0d8d9766e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3914bffa8e24e51aabe2b7b58ff65b1, user_id=9438c91ac19c49b4a711d4d9aafdeac5, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:38.365520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e0ae5a-6da5-4016-b48f-cb0d8d9766e8 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:38.378271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1216/2459] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:08:38 2022] PUT /identity/v3/projects/c3914bffa8e24e51aabe2b7b58ff65b1/users/9438c91ac19c49b4a711d4d9aafdeac5/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:38.388334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbc045-93e7-42ee-b1db-c938510debcd None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:38.388649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbc045-93e7-42ee-b1db-c938510debcd None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:38.388922 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbc045-93e7-42ee-b1db-c938510debcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:38.417976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-52cbc045-93e7-42ee-b1db-c938510debcd None None] MFA Rules not processed for user `9438c91ac19c49b4a711d4d9aafdeac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:08:38.419172 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-52cbc045-93e7-42ee-b1db-c938510debcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:38.482316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1244/2460] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:08:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 13:08:38.487443 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0ae5a-6da5-4016-b48f-cb0d8d9766e8 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:38.546640 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0ae5a-6da5-4016-b48f-cb0d8d9766e8 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:38.551376 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801afa7-3988-4239-8305-483b1826bb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:38.551984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801afa7-3988-4239-8305-483b1826bb88 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:38.552262 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801afa7-3988-4239-8305-483b1826bb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:38.552565 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801afa7-3988-4239-8305-483b1826bb88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:38.553218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f801afa7-3988-4239-8305-483b1826bb88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:38.555117 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f801afa7-3988-4239-8305-483b1826bb88 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:38.580827 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1217/2461] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:08:38 2022] POST /identity/v3/projects => generated 375 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:08:38.584813 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbc045-93e7-42ee-b1db-c938510debcd None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:38.595790 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbc045-93e7-42ee-b1db-c938510debcd None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:38.649980 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7d6e2-dd52-4632-8bf4-4e458158ea72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:38.650770 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7d6e2-dd52-4632-8bf4-4e458158ea72 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:38.651079 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7d6e2-dd52-4632-8bf4-4e458158ea72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:38.651354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7d6e2-dd52-4632-8bf4-4e458158ea72 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:38.652017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7d6e2-dd52-4632-8bf4-4e458158ea72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:38.653950 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7d6e2-dd52-4632-8bf4-4e458158ea72 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:38.675271 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1245/2462] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:08:38 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:08:38.679007 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801afa7-3988-4239-8305-483b1826bb88 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:38.689153 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801afa7-3988-4239-8305-483b1826bb88 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:38.693703 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229df3c-016d-4d12-bafc-10348f431f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:38.694296 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229df3c-016d-4d12-bafc-10348f431f9c admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:38.694572 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229df3c-016d-4d12-bafc-10348f431f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:38.694842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229df3c-016d-4d12-bafc-10348f431f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:38.695529 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229df3c-016d-4d12-bafc-10348f431f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:38.697151 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229df3c-016d-4d12-bafc-10348f431f9c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:38.702167 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:38.708511 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1218/2463] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:08:38.712450 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7d6e2-dd52-4632-8bf4-4e458158ea72 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:38.723561 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7d6e2-dd52-4632-8bf4-4e458158ea72 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:38.728379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8cc49-67ca-42a5-9157-1d61de659c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:38.729025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8cc49-67ca-42a5-9157-1d61de659c3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:38.729304 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8cc49-67ca-42a5-9157-1d61de659c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:38.729602 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8cc49-67ca-42a5-9157-1d61de659c3c admin admin] PATH_INFO: `/v3/projects/7592555fb9eb4bb78c855d9de149724e/users/7a915833d2874a338266a442abc01dce/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:38.744697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df8cc49-67ca-42a5-9157-1d61de659c3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7592555fb9eb4bb78c855d9de149724e, user_id=7a915833d2874a338266a442abc01dce, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:38.747381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df8cc49-67ca-42a5-9157-1d61de659c3c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:38.760048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1246/2464] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:08:38 2022] PUT /identity/v3/projects/7592555fb9eb4bb78c855d9de149724e/users/7a915833d2874a338266a442abc01dce/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:38.763989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229df3c-016d-4d12-bafc-10348f431f9c admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:38.773891 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229df3c-016d-4d12-bafc-10348f431f9c admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:38.828653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c66280-f090-4f35-af84-e86c29f2cdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:38.829254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c66280-f090-4f35-af84-e86c29f2cdff admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:38.829632 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c66280-f090-4f35-af84-e86c29f2cdff admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:38.829799 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c66280-f090-4f35-af84-e86c29f2cdff admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:38.830459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c66280-f090-4f35-af84-e86c29f2cdff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:38.832141 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c66280-f090-4f35-af84-e86c29f2cdff admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:38.844412 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1219/2465] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:08:38.847992 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8cc49-67ca-42a5-9157-1d61de659c3c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:38.858810 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8cc49-67ca-42a5-9157-1d61de659c3c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:38.864205 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190afb2f-5af3-4ca1-8e78-39488ac9f2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:38.864899 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190afb2f-5af3-4ca1-8e78-39488ac9f2b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:38.865194 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190afb2f-5af3-4ca1-8e78-39488ac9f2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:38.865570 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190afb2f-5af3-4ca1-8e78-39488ac9f2b9 admin admin] PATH_INFO: `/v3/projects/7592555fb9eb4bb78c855d9de149724e/users/7a915833d2874a338266a442abc01dce/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:38.867528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190afb2f-5af3-4ca1-8e78-39488ac9f2b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7592555fb9eb4bb78c855d9de149724e, user_id=7a915833d2874a338266a442abc01dce, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:38.870201 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190afb2f-5af3-4ca1-8e78-39488ac9f2b9 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:38.880651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1247/2466] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:08:38 2022] PUT /identity/v3/projects/7592555fb9eb4bb78c855d9de149724e/users/7a915833d2874a338266a442abc01dce/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:38.891198 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b0e47-0def-4197-982c-c3df1920b582 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:38.891672 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b0e47-0def-4197-982c-c3df1920b582 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:38.891822 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b0e47-0def-4197-982c-c3df1920b582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:38.920560 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-216b0e47-0def-4197-982c-c3df1920b582 None None] MFA Rules not processed for user `7a915833d2874a338266a442abc01dce`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:08:38.921767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-216b0e47-0def-4197-982c-c3df1920b582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:38.992214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1220/2467] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:08:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:08:39.002658 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190afb2f-5af3-4ca1-8e78-39488ac9f2b9 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:39.004511 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-190afb2f-5af3-4ca1-8e78-39488ac9f2b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:39.035938 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:39.065731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190afb2f-5af3-4ca1-8e78-39488ac9f2b9 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:39.069455 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1851c2f-ef5c-4fce-91ff-42a8504c7f21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:39.069971 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1851c2f-ef5c-4fce-91ff-42a8504c7f21 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:39.070252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1851c2f-ef5c-4fce-91ff-42a8504c7f21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:39.070522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1851c2f-ef5c-4fce-91ff-42a8504c7f21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:39.125476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1851c2f-ef5c-4fce-91ff-42a8504c7f21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:39.126894 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1851c2f-ef5c-4fce-91ff-42a8504c7f21 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:39.135577 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1248/2468] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:08:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:08:41.647320 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b0e47-0def-4197-982c-c3df1920b582 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:41.709279 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b0e47-0def-4197-982c-c3df1920b582 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:41.713769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cfb418-86cd-40da-a631-c97f79869417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:41.714396 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cfb418-86cd-40da-a631-c97f79869417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:41.714675 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cfb418-86cd-40da-a631-c97f79869417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:41.714947 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cfb418-86cd-40da-a631-c97f79869417 admin admin] PATH_INFO: `/v3/users/9438c91ac19c49b4a711d4d9aafdeac5` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:41.716083 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cfb418-86cd-40da-a631-c97f79869417 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9438c91ac19c49b4a711d4d9aafdeac5)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:41.718576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cfb418-86cd-40da-a631-c97f79869417 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:41.740898 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:41.765027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c7cfb418-86cd-40da-a631-c97f79869417 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9438c91ac19c49b4a711d4d9aafdeac5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:41.771814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c7cfb418-86cd-40da-a631-c97f79869417 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9438c91ac19c49b4a711d4d9aafdeac5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:41.784261 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c7cfb418-86cd-40da-a631-c97f79869417 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9438c91ac19c49b4a711d4d9aafdeac5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:41.784969 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c7cfb418-86cd-40da-a631-c97f79869417 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9438c91ac19c49b4a711d4d9aafdeac5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:41.785563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c7cfb418-86cd-40da-a631-c97f79869417 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9438c91ac19c49b4a711d4d9aafdeac5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:41.799152 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1221/2469] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:08:41 2022] DELETE /identity/v3/users/9438c91ac19c49b4a711d4d9aafdeac5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:41.803568 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448f9ee-d772-4ac3-9274-b73dc981ae55 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:41.804043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448f9ee-d772-4ac3-9274-b73dc981ae55 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:41.804142 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448f9ee-d772-4ac3-9274-b73dc981ae55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:41.822802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6448f9ee-d772-4ac3-9274-b73dc981ae55 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:08:41.823989 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6448f9ee-d772-4ac3-9274-b73dc981ae55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:41.881818 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1249/2470] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:08:41.891893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cfb418-86cd-40da-a631-c97f79869417 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:41.893321 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c7cfb418-86cd-40da-a631-c97f79869417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:41.948803 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cfb418-86cd-40da-a631-c97f79869417 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:41.952990 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18aa311-4910-415d-9170-e4ecde6a093f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:41.953682 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18aa311-4910-415d-9170-e4ecde6a093f service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:41.953962 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18aa311-4910-415d-9170-e4ecde6a093f service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:41.954238 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18aa311-4910-415d-9170-e4ecde6a093f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:41.963098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18aa311-4910-415d-9170-e4ecde6a093f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:41.964557 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18aa311-4910-415d-9170-e4ecde6a093f service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:41.974453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1222/2471] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:08:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:08:42.239568 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448f9ee-d772-4ac3-9274-b73dc981ae55 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:42.241009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6448f9ee-d772-4ac3-9274-b73dc981ae55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:42.253846 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448f9ee-d772-4ac3-9274-b73dc981ae55 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:42.258447 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776ef22-e269-4f26-af9e-96bcb5d89139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:42.259067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776ef22-e269-4f26-af9e-96bcb5d89139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:42.259353 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776ef22-e269-4f26-af9e-96bcb5d89139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:42.259631 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776ef22-e269-4f26-af9e-96bcb5d89139 admin admin] PATH_INFO: `/v3/projects/c3914bffa8e24e51aabe2b7b58ff65b1` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:42.261195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776ef22-e269-4f26-af9e-96bcb5d89139 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3914bffa8e24e51aabe2b7b58ff65b1)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:42.263127 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776ef22-e269-4f26-af9e-96bcb5d89139 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:42.309285 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3776ef22-e269-4f26-af9e-96bcb5d89139 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3914bffa8e24e51aabe2b7b58ff65b1', 'request_id': 'req-3776ef22-e269-4f26-af9e-96bcb5d89139'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:42.315158 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3776ef22-e269-4f26-af9e-96bcb5d89139 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3914bffa8e24e51aabe2b7b58ff65b1', 'request_id': 'req-3776ef22-e269-4f26-af9e-96bcb5d89139'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:42.322722 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3776ef22-e269-4f26-af9e-96bcb5d89139 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3914bffa8e24e51aabe2b7b58ff65b1', 'request_id': 'req-3776ef22-e269-4f26-af9e-96bcb5d89139'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:42.334011 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3776ef22-e269-4f26-af9e-96bcb5d89139 admin admin] The token cache is being invalidate because project c3914bffa8e24e51aabe2b7b58ff65b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:08:42.334296 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3776ef22-e269-4f26-af9e-96bcb5d89139 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:42.334918 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3776ef22-e269-4f26-af9e-96bcb5d89139 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:42.336414 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1250/2472] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:08:42 2022] DELETE /identity/v3/projects/c3914bffa8e24e51aabe2b7b58ff65b1 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:42.340341 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18aa311-4910-415d-9170-e4ecde6a093f service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:42.350448 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18aa311-4910-415d-9170-e4ecde6a093f service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:42.404249 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4746c-62ae-46ba-b18e-8c8269ea0263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:42.404918 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4746c-62ae-46ba-b18e-8c8269ea0263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:42.405571 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4746c-62ae-46ba-b18e-8c8269ea0263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:42.405845 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4746c-62ae-46ba-b18e-8c8269ea0263 admin admin] PATH_INFO: `/v3/users/7a915833d2874a338266a442abc01dce` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:42.407014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4746c-62ae-46ba-b18e-8c8269ea0263 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a915833d2874a338266a442abc01dce)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:42.409767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4746c-62ae-46ba-b18e-8c8269ea0263 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:42.455706 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:42.457241 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-59f4746c-62ae-46ba-b18e-8c8269ea0263 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a915833d2874a338266a442abc01dce'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:42.466006 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-59f4746c-62ae-46ba-b18e-8c8269ea0263 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a915833d2874a338266a442abc01dce'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:42.477995 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-59f4746c-62ae-46ba-b18e-8c8269ea0263 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a915833d2874a338266a442abc01dce'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:42.478691 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-59f4746c-62ae-46ba-b18e-8c8269ea0263 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a915833d2874a338266a442abc01dce'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:42.479292 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-59f4746c-62ae-46ba-b18e-8c8269ea0263 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a915833d2874a338266a442abc01dce'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:42.491475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1223/2473] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:08:42 2022] DELETE /identity/v3/users/7a915833d2874a338266a442abc01dce => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:42.763537 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776ef22-e269-4f26-af9e-96bcb5d89139 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:42.765145 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3776ef22-e269-4f26-af9e-96bcb5d89139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:42.774569 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:42.778855 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776ef22-e269-4f26-af9e-96bcb5d89139 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:42.836594 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473b195-04f2-4011-abb6-047583197c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:42.837221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473b195-04f2-4011-abb6-047583197c1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:42.837495 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473b195-04f2-4011-abb6-047583197c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:42.837764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473b195-04f2-4011-abb6-047583197c1b admin admin] PATH_INFO: `/v3/projects/7592555fb9eb4bb78c855d9de149724e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:42.838898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473b195-04f2-4011-abb6-047583197c1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7592555fb9eb4bb78c855d9de149724e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:42.840821 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473b195-04f2-4011-abb6-047583197c1b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:42.886805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0473b195-04f2-4011-abb6-047583197c1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7592555fb9eb4bb78c855d9de149724e', 'request_id': 'req-0473b195-04f2-4011-abb6-047583197c1b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:42.892248 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0473b195-04f2-4011-abb6-047583197c1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7592555fb9eb4bb78c855d9de149724e', 'request_id': 'req-0473b195-04f2-4011-abb6-047583197c1b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:42.899565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0473b195-04f2-4011-abb6-047583197c1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7592555fb9eb4bb78c855d9de149724e', 'request_id': 'req-0473b195-04f2-4011-abb6-047583197c1b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:42.911337 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0473b195-04f2-4011-abb6-047583197c1b admin admin] The token cache is being invalidate because project 7592555fb9eb4bb78c855d9de149724e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:08:42.911629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0473b195-04f2-4011-abb6-047583197c1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:42.912267 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0473b195-04f2-4011-abb6-047583197c1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:42.913799 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1251/2474] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:08:42 2022] DELETE /identity/v3/projects/7592555fb9eb4bb78c855d9de149724e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:42.918689 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f01b1-4739-41e2-95bb-8936a82b7026 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:42.919185 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f01b1-4739-41e2-95bb-8936a82b7026 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:42.919277 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f01b1-4739-41e2-95bb-8936a82b7026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:42.938840 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-485f01b1-4739-41e2-95bb-8936a82b7026 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:08:42.940019 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-485f01b1-4739-41e2-95bb-8936a82b7026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:42.998624 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1224/2475] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:08:43.007373 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd000fb4-2d72-41f4-84ee-1bef4cc61096 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:43.007661 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd000fb4-2d72-41f4-84ee-1bef4cc61096 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:43.007939 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd000fb4-2d72-41f4-84ee-1bef4cc61096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:43.027316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-fd000fb4-2d72-41f4-84ee-1bef4cc61096 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:08:43.028555 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fd000fb4-2d72-41f4-84ee-1bef4cc61096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:43.037337 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1252/2476] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:08:43.041654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f01b1-4739-41e2-95bb-8936a82b7026 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:43.052642 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f01b1-4739-41e2-95bb-8936a82b7026 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:43.058978 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ab788-b07c-41c4-8667-5ee5513c37bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:43.060204 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ab788-b07c-41c4-8667-5ee5513c37bd admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:43.060624 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ab788-b07c-41c4-8667-5ee5513c37bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:43.060958 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ab788-b07c-41c4-8667-5ee5513c37bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:43.061833 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ab788-b07c-41c4-8667-5ee5513c37bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:08:43.062143 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ab788-b07c-41c4-8667-5ee5513c37bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:43.064257 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ab788-b07c-41c4-8667-5ee5513c37bd admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:43.077030 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1225/2477] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 13:08:43.081312 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd000fb4-2d72-41f4-84ee-1bef4cc61096 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:43.090515 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd000fb4-2d72-41f4-84ee-1bef4cc61096 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:43.095632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f2bbd-b4fc-4078-b1a1-9af092c70c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:43.096229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f2bbd-b4fc-4078-b1a1-9af092c70c94 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:43.096544 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f2bbd-b4fc-4078-b1a1-9af092c70c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:43.096818 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f2bbd-b4fc-4078-b1a1-9af092c70c94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:43.097469 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f2bbd-b4fc-4078-b1a1-9af092c70c94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:43.099344 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f2bbd-b4fc-4078-b1a1-9af092c70c94 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:43.127377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1253/2478] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:08:43 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:08:43.131504 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ab788-b07c-41c4-8667-5ee5513c37bd admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:43.141656 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ab788-b07c-41c4-8667-5ee5513c37bd admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:43.196785 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9730ca2-6629-4038-ac38-9fb8e684af4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:43.197387 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9730ca2-6629-4038-ac38-9fb8e684af4a admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:43.197799 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9730ca2-6629-4038-ac38-9fb8e684af4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:43.198070 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9730ca2-6629-4038-ac38-9fb8e684af4a admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:43.198724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9730ca2-6629-4038-ac38-9fb8e684af4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:43.200716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9730ca2-6629-4038-ac38-9fb8e684af4a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:43.221923 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1226/2479] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:08:43 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:08:43.226265 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f2bbd-b4fc-4078-b1a1-9af092c70c94 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:43.237144 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f2bbd-b4fc-4078-b1a1-9af092c70c94 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:43.242073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a8197-108b-4a5f-889d-5554ad1ccdd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:43.242907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a8197-108b-4a5f-889d-5554ad1ccdd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:43.242999 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a8197-108b-4a5f-889d-5554ad1ccdd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:43.243208 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a8197-108b-4a5f-889d-5554ad1ccdd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:43.244028 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332a8197-108b-4a5f-889d-5554ad1ccdd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:43.245653 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332a8197-108b-4a5f-889d-5554ad1ccdd1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:43.259124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1254/2480] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:08:43.260523 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9730ca2-6629-4038-ac38-9fb8e684af4a admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:43.270667 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9730ca2-6629-4038-ac38-9fb8e684af4a admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:43.275289 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128a5fd-e729-4b1b-aa17-813996a30b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:43.275881 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128a5fd-e729-4b1b-aa17-813996a30b72 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:43.276159 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128a5fd-e729-4b1b-aa17-813996a30b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:43.276484 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128a5fd-e729-4b1b-aa17-813996a30b72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:43.277164 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128a5fd-e729-4b1b-aa17-813996a30b72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:43.278770 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128a5fd-e729-4b1b-aa17-813996a30b72 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:43.289959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1227/2481] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:08:43.294190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a8197-108b-4a5f-889d-5554ad1ccdd1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:43.304891 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a8197-108b-4a5f-889d-5554ad1ccdd1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:43.309585 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336c52c-4998-46a8-b636-1936b395f239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:43.310235 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336c52c-4998-46a8-b636-1936b395f239 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:43.310518 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336c52c-4998-46a8-b636-1936b395f239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:43.310797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336c52c-4998-46a8-b636-1936b395f239 admin admin] PATH_INFO: `/v3/projects/93a704f90e0740359742bf852753073a/users/76a798c1fd45420e8ff1b2c3c857c010/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:43.325047 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336c52c-4998-46a8-b636-1936b395f239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a704f90e0740359742bf852753073a, user_id=76a798c1fd45420e8ff1b2c3c857c010, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:43.328194 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336c52c-4998-46a8-b636-1936b395f239 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:43.341171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1255/2482] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:08:43 2022] PUT /identity/v3/projects/93a704f90e0740359742bf852753073a/users/76a798c1fd45420e8ff1b2c3c857c010/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:43.356684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141c391-c35b-482c-b357-cac1f2af8d73 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:43.356947 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141c391-c35b-482c-b357-cac1f2af8d73 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:43.357219 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141c391-c35b-482c-b357-cac1f2af8d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:43.386688 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-5141c391-c35b-482c-b357-cac1f2af8d73 None None] MFA Rules not processed for user `76a798c1fd45420e8ff1b2c3c857c010`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:08:43.387879 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5141c391-c35b-482c-b357-cac1f2af8d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:43.404166 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:43.450811 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1228/2483] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:08:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:08:43.455732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336c52c-4998-46a8-b636-1936b395f239 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:43.488529 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:43.518228 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336c52c-4998-46a8-b636-1936b395f239 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:43.522753 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9430955-882e-477a-814a-a20387d11acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:43.523354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9430955-882e-477a-814a-a20387d11acf admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:43.523636 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9430955-882e-477a-814a-a20387d11acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:43.523905 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9430955-882e-477a-814a-a20387d11acf admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:43.524599 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9430955-882e-477a-814a-a20387d11acf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:43.526473 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9430955-882e-477a-814a-a20387d11acf admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:43.552242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1256/2484] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:08:43 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:08:43.556181 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141c391-c35b-482c-b357-cac1f2af8d73 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:43.566805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141c391-c35b-482c-b357-cac1f2af8d73 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:43.621552 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e6eb9-058e-4631-ad24-340b3b32369b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:43.622157 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e6eb9-058e-4631-ad24-340b3b32369b admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:43.622435 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e6eb9-058e-4631-ad24-340b3b32369b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:43.622706 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e6eb9-058e-4631-ad24-340b3b32369b admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:43.623350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e6eb9-058e-4631-ad24-340b3b32369b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:43.625309 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e6eb9-058e-4631-ad24-340b3b32369b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:43.646096 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1229/2485] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:08:43 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:08:43.650659 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9430955-882e-477a-814a-a20387d11acf admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:43.665446 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9430955-882e-477a-814a-a20387d11acf admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:43.670229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba310565-91e7-47f1-a98b-9211f56f7621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:43.670850 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba310565-91e7-47f1-a98b-9211f56f7621 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:43.671140 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba310565-91e7-47f1-a98b-9211f56f7621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:43.671418 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba310565-91e7-47f1-a98b-9211f56f7621 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:43.672116 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba310565-91e7-47f1-a98b-9211f56f7621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:43.673771 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba310565-91e7-47f1-a98b-9211f56f7621 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:43.684730 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1257/2486] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:08:43.689355 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e6eb9-058e-4631-ad24-340b3b32369b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:43.700409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e6eb9-058e-4631-ad24-340b3b32369b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:43.706078 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f581bc9-eb28-4621-9b45-237454eeb1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:43.706696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f581bc9-eb28-4621-9b45-237454eeb1b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:43.706980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f581bc9-eb28-4621-9b45-237454eeb1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:43.707255 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f581bc9-eb28-4621-9b45-237454eeb1b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:43.707950 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f581bc9-eb28-4621-9b45-237454eeb1b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:43.709671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f581bc9-eb28-4621-9b45-237454eeb1b9 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:43.720160 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1230/2487] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:08:43.724202 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba310565-91e7-47f1-a98b-9211f56f7621 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:43.734515 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba310565-91e7-47f1-a98b-9211f56f7621 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:43.739182 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc744cdf-939d-430c-8d10-b263a54a13c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:43.739832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc744cdf-939d-430c-8d10-b263a54a13c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:43.740113 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc744cdf-939d-430c-8d10-b263a54a13c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:43.740428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc744cdf-939d-430c-8d10-b263a54a13c6 admin admin] PATH_INFO: `/v3/projects/573da2ba76cb42de8f9ea68226ac98a7/users/6566bc3a884f46e086ff93c7e48962c5/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:43.756160 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc744cdf-939d-430c-8d10-b263a54a13c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=573da2ba76cb42de8f9ea68226ac98a7, user_id=6566bc3a884f46e086ff93c7e48962c5, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:43.758970 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc744cdf-939d-430c-8d10-b263a54a13c6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:43.772178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1258/2488] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:08:43 2022] PUT /identity/v3/projects/573da2ba76cb42de8f9ea68226ac98a7/users/6566bc3a884f46e086ff93c7e48962c5/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:43.783215 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abd7ae-d438-4fbe-bec2-31cc06142390 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:43.783502 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abd7ae-d438-4fbe-bec2-31cc06142390 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:43.783777 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abd7ae-d438-4fbe-bec2-31cc06142390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:43.813302 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c1abd7ae-d438-4fbe-bec2-31cc06142390 None None] MFA Rules not processed for user `6566bc3a884f46e086ff93c7e48962c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:08:43.814522 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c1abd7ae-d438-4fbe-bec2-31cc06142390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:43.876895 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1231/2489] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:08:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:08:43.881308 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc744cdf-939d-430c-8d10-b263a54a13c6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:43.941721 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc744cdf-939d-430c-8d10-b263a54a13c6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:43.946341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb5625-83d3-4177-b92d-12411c839455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:43.946951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb5625-83d3-4177-b92d-12411c839455 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:43.947236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb5625-83d3-4177-b92d-12411c839455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:43.947515 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb5625-83d3-4177-b92d-12411c839455 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:43.948214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb5625-83d3-4177-b92d-12411c839455 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:43.950182 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb5625-83d3-4177-b92d-12411c839455 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:43.977013 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1259/2490] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:08:43 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:08:43.981035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abd7ae-d438-4fbe-bec2-31cc06142390 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:43.992093 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abd7ae-d438-4fbe-bec2-31cc06142390 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:44.018157 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:44.054096 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acb9b9-9076-4011-944a-bdf350592482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:44.054714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acb9b9-9076-4011-944a-bdf350592482 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:44.055000 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acb9b9-9076-4011-944a-bdf350592482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:44.055278 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acb9b9-9076-4011-944a-bdf350592482 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:44.056054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37acb9b9-9076-4011-944a-bdf350592482 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:44.058024 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37acb9b9-9076-4011-944a-bdf350592482 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:44.078445 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1232/2491] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:08:43 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:08:44.082441 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb5625-83d3-4177-b92d-12411c839455 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:44.092716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb5625-83d3-4177-b92d-12411c839455 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:44.097343 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac312d4a-f684-4a7c-a80f-1b321c15347b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:44.097941 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac312d4a-f684-4a7c-a80f-1b321c15347b admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:44.098217 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac312d4a-f684-4a7c-a80f-1b321c15347b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:44.098486 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac312d4a-f684-4a7c-a80f-1b321c15347b admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:44.099168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac312d4a-f684-4a7c-a80f-1b321c15347b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:44.101055 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac312d4a-f684-4a7c-a80f-1b321c15347b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:44.111851 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1260/2492] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:08:44.115586 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acb9b9-9076-4011-944a-bdf350592482 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:44.125753 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acb9b9-9076-4011-944a-bdf350592482 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:44.130224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9553617-e217-4df0-af00-9916c7899836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:44.130844 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9553617-e217-4df0-af00-9916c7899836 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:44.131135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9553617-e217-4df0-af00-9916c7899836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:44.131388 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9553617-e217-4df0-af00-9916c7899836 admin admin] PATH_INFO: `/v3/projects/80a66ad9dd584d14be9d285b8a63d25a/users/eb68525e5d604694b88b7fa2f86de2cb/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:44.145550 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9553617-e217-4df0-af00-9916c7899836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80a66ad9dd584d14be9d285b8a63d25a, user_id=eb68525e5d604694b88b7fa2f86de2cb, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:44.148209 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9553617-e217-4df0-af00-9916c7899836 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:44.161123 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1233/2493] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:08:44 2022] PUT /identity/v3/projects/80a66ad9dd584d14be9d285b8a63d25a/users/eb68525e5d604694b88b7fa2f86de2cb/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:08:44.164947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac312d4a-f684-4a7c-a80f-1b321c15347b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:44.175166 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac312d4a-f684-4a7c-a80f-1b321c15347b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:44.222743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:44.230802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea4c82-ea38-4db1-a6ad-8a8a0f1ceec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:44.231415 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea4c82-ea38-4db1-a6ad-8a8a0f1ceec1 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:44.231775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea4c82-ea38-4db1-a6ad-8a8a0f1ceec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:44.232052 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea4c82-ea38-4db1-a6ad-8a8a0f1ceec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:44.232784 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cea4c82-ea38-4db1-a6ad-8a8a0f1ceec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:44.234858 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cea4c82-ea38-4db1-a6ad-8a8a0f1ceec1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:44.244576 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1261/2494] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:08:44.248589 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9553617-e217-4df0-af00-9916c7899836 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:44.259306 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9553617-e217-4df0-af00-9916c7899836 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:44.264077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7ecb9-a8a3-4735-a1be-b4bbe083b585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:44.264932 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7ecb9-a8a3-4735-a1be-b4bbe083b585 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:44.265235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7ecb9-a8a3-4735-a1be-b4bbe083b585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:44.265528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7ecb9-a8a3-4735-a1be-b4bbe083b585 admin admin] PATH_INFO: `/v3/projects/80a66ad9dd584d14be9d285b8a63d25a/users/eb68525e5d604694b88b7fa2f86de2cb/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:44.267443 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7ecb9-a8a3-4735-a1be-b4bbe083b585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80a66ad9dd584d14be9d285b8a63d25a, user_id=eb68525e5d604694b88b7fa2f86de2cb, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:44.270065 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7ecb9-a8a3-4735-a1be-b4bbe083b585 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:44.280024 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1234/2495] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:08:44 2022] PUT /identity/v3/projects/80a66ad9dd584d14be9d285b8a63d25a/users/eb68525e5d604694b88b7fa2f86de2cb/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:44.290181 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab2104-9fa6-46ac-86f3-c57c25e9dacb None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:44.290472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab2104-9fa6-46ac-86f3-c57c25e9dacb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:44.290752 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab2104-9fa6-46ac-86f3-c57c25e9dacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:44.319792 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a7ab2104-9fa6-46ac-86f3-c57c25e9dacb None None] MFA Rules not processed for user `eb68525e5d604694b88b7fa2f86de2cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:08:44.321056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a7ab2104-9fa6-46ac-86f3-c57c25e9dacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:44.388631 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1262/2496] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:08:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:08:44.399959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7ecb9-a8a3-4735-a1be-b4bbe083b585 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:44.401648 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c7d7ecb9-a8a3-4735-a1be-b4bbe083b585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:44.458917 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7ecb9-a8a3-4735-a1be-b4bbe083b585 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:44.463006 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70da3f-2c82-44f2-8920-12a98c0ab3d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:44.463606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70da3f-2c82-44f2-8920-12a98c0ab3d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:44.463881 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70da3f-2c82-44f2-8920-12a98c0ab3d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:44.464148 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70da3f-2c82-44f2-8920-12a98c0ab3d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:44.518075 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70da3f-2c82-44f2-8920-12a98c0ab3d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:44.519483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70da3f-2c82-44f2-8920-12a98c0ab3d2 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:44.528059 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1235/2497] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:08:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:08:46.008689 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab2104-9fa6-46ac-86f3-c57c25e9dacb None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:46.018874 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab2104-9fa6-46ac-86f3-c57c25e9dacb None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:46.022427 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7076c29-23a4-411e-b999-267e0c26935a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:46.023041 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7076c29-23a4-411e-b999-267e0c26935a service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:46.023319 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7076c29-23a4-411e-b999-267e0c26935a service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:46.023622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7076c29-23a4-411e-b999-267e0c26935a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:46.079241 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7076c29-23a4-411e-b999-267e0c26935a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:46.080752 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7076c29-23a4-411e-b999-267e0c26935a service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:46.089641 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1263/2498] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:08:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 13:08:46.602487 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70da3f-2c82-44f2-8920-12a98c0ab3d2 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:46.611113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70da3f-2c82-44f2-8920-12a98c0ab3d2 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:46.614657 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a445b9-d0ca-4981-aabe-5a8c7ff16494 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:46.615275 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a445b9-d0ca-4981-aabe-5a8c7ff16494 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:46.615556 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a445b9-d0ca-4981-aabe-5a8c7ff16494 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:46.615863 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a445b9-d0ca-4981-aabe-5a8c7ff16494 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:46.625815 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a445b9-d0ca-4981-aabe-5a8c7ff16494 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:46.627255 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a445b9-d0ca-4981-aabe-5a8c7ff16494 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:46.636958 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1236/2499] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:08:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:08:50.482143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7076c29-23a4-411e-b999-267e0c26935a service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:50.543259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7076c29-23a4-411e-b999-267e0c26935a service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:50.547717 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be190d83-bc13-49ba-8d81-24b46981d936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:50.548369 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be190d83-bc13-49ba-8d81-24b46981d936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:50.548651 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be190d83-bc13-49ba-8d81-24b46981d936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:50.548923 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be190d83-bc13-49ba-8d81-24b46981d936 admin admin] PATH_INFO: `/v3/users/76a798c1fd45420e8ff1b2c3c857c010` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:50.550069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be190d83-bc13-49ba-8d81-24b46981d936 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76a798c1fd45420e8ff1b2c3c857c010)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:50.551964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be190d83-bc13-49ba-8d81-24b46981d936 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:50.567602 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:50.595860 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-be190d83-bc13-49ba-8d81-24b46981d936 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76a798c1fd45420e8ff1b2c3c857c010'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:50.608033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-be190d83-bc13-49ba-8d81-24b46981d936 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76a798c1fd45420e8ff1b2c3c857c010'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:50.608732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-be190d83-bc13-49ba-8d81-24b46981d936 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76a798c1fd45420e8ff1b2c3c857c010'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:50.609332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-be190d83-bc13-49ba-8d81-24b46981d936 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76a798c1fd45420e8ff1b2c3c857c010'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:50.617485 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-be190d83-bc13-49ba-8d81-24b46981d936 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76a798c1fd45420e8ff1b2c3c857c010'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:50.630347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1264/2500] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:08:50 2022] DELETE /identity/v3/users/76a798c1fd45420e8ff1b2c3c857c010 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:50.634797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49dd14-fe60-4d74-a213-2007bbb6200a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:50.635248 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49dd14-fe60-4d74-a213-2007bbb6200a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:50.635412 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49dd14-fe60-4d74-a213-2007bbb6200a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:50.654508 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-6c49dd14-fe60-4d74-a213-2007bbb6200a None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:08:50.655697 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6c49dd14-fe60-4d74-a213-2007bbb6200a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:50.668125 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:50.713345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1237/2501] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:08:50.726238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be190d83-bc13-49ba-8d81-24b46981d936 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:50.727732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-be190d83-bc13-49ba-8d81-24b46981d936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:50.785817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be190d83-bc13-49ba-8d81-24b46981d936 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:50.789398 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64be655-9f10-4d27-9531-c2a3b4e4631d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:50.790009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64be655-9f10-4d27-9531-c2a3b4e4631d service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:50.790286 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64be655-9f10-4d27-9531-c2a3b4e4631d service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:50.790558 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64be655-9f10-4d27-9531-c2a3b4e4631d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:50.799808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64be655-9f10-4d27-9531-c2a3b4e4631d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:50.801249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64be655-9f10-4d27-9531-c2a3b4e4631d service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:50.810543 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1265/2502] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:08:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:08:51.080610 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49dd14-fe60-4d74-a213-2007bbb6200a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:51.082296 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6c49dd14-fe60-4d74-a213-2007bbb6200a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:51.098651 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49dd14-fe60-4d74-a213-2007bbb6200a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:51.105242 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d933d5-3d77-4d12-89db-c02f430e49d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:51.106266 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d933d5-3d77-4d12-89db-c02f430e49d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:51.106792 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d933d5-3d77-4d12-89db-c02f430e49d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:51.107248 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d933d5-3d77-4d12-89db-c02f430e49d1 admin admin] PATH_INFO: `/v3/projects/93a704f90e0740359742bf852753073a` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:51.109763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d933d5-3d77-4d12-89db-c02f430e49d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93a704f90e0740359742bf852753073a)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:51.112242 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d933d5-3d77-4d12-89db-c02f430e49d1 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:51.158829 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-b4d933d5-3d77-4d12-89db-c02f430e49d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93a704f90e0740359742bf852753073a', 'request_id': 'req-b4d933d5-3d77-4d12-89db-c02f430e49d1'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:51.164281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-b4d933d5-3d77-4d12-89db-c02f430e49d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93a704f90e0740359742bf852753073a', 'request_id': 'req-b4d933d5-3d77-4d12-89db-c02f430e49d1'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:51.172813 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-b4d933d5-3d77-4d12-89db-c02f430e49d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93a704f90e0740359742bf852753073a', 'request_id': 'req-b4d933d5-3d77-4d12-89db-c02f430e49d1'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:51.183513 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-b4d933d5-3d77-4d12-89db-c02f430e49d1 admin admin] The token cache is being invalidate because project 93a704f90e0740359742bf852753073a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:08:51.183793 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-b4d933d5-3d77-4d12-89db-c02f430e49d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:51.184434 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-b4d933d5-3d77-4d12-89db-c02f430e49d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:51.185919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1238/2503] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:08:51 2022] DELETE /identity/v3/projects/93a704f90e0740359742bf852753073a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:51.189913 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64be655-9f10-4d27-9531-c2a3b4e4631d service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:51.199653 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64be655-9f10-4d27-9531-c2a3b4e4631d service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:51.255149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57201888-ee09-4f74-8ca8-65979f371996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:51.255783 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57201888-ee09-4f74-8ca8-65979f371996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:51.256065 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57201888-ee09-4f74-8ca8-65979f371996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:51.256377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57201888-ee09-4f74-8ca8-65979f371996 admin admin] PATH_INFO: `/v3/users/6566bc3a884f46e086ff93c7e48962c5` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:51.257574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57201888-ee09-4f74-8ca8-65979f371996 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6566bc3a884f46e086ff93c7e48962c5)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:51.259506 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57201888-ee09-4f74-8ca8-65979f371996 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:51.298323 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:51.305528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-57201888-ee09-4f74-8ca8-65979f371996 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6566bc3a884f46e086ff93c7e48962c5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:51.317216 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-57201888-ee09-4f74-8ca8-65979f371996 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6566bc3a884f46e086ff93c7e48962c5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:51.317883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-57201888-ee09-4f74-8ca8-65979f371996 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6566bc3a884f46e086ff93c7e48962c5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:51.318479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-57201888-ee09-4f74-8ca8-65979f371996 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6566bc3a884f46e086ff93c7e48962c5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:51.325623 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-57201888-ee09-4f74-8ca8-65979f371996 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6566bc3a884f46e086ff93c7e48962c5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:51.337415 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1266/2504] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:08:51 2022] DELETE /identity/v3/users/6566bc3a884f46e086ff93c7e48962c5 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:51.605271 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d933d5-3d77-4d12-89db-c02f430e49d1 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:51.606738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b4d933d5-3d77-4d12-89db-c02f430e49d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:51.623644 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d933d5-3d77-4d12-89db-c02f430e49d1 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:51.681797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700e541-f5a7-463a-a2f8-20b02d4e64fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:51.682463 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700e541-f5a7-463a-a2f8-20b02d4e64fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:51.682747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700e541-f5a7-463a-a2f8-20b02d4e64fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:51.683027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700e541-f5a7-463a-a2f8-20b02d4e64fd admin admin] PATH_INFO: `/v3/projects/573da2ba76cb42de8f9ea68226ac98a7` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:51.684199 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700e541-f5a7-463a-a2f8-20b02d4e64fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=573da2ba76cb42de8f9ea68226ac98a7)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:51.686148 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700e541-f5a7-463a-a2f8-20b02d4e64fd admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:51.729296 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:08:51.732483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-b700e541-f5a7-463a-a2f8-20b02d4e64fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '573da2ba76cb42de8f9ea68226ac98a7', 'request_id': 'req-b700e541-f5a7-463a-a2f8-20b02d4e64fd'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:51.739143 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-b700e541-f5a7-463a-a2f8-20b02d4e64fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '573da2ba76cb42de8f9ea68226ac98a7', 'request_id': 'req-b700e541-f5a7-463a-a2f8-20b02d4e64fd'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:51.745796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-b700e541-f5a7-463a-a2f8-20b02d4e64fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '573da2ba76cb42de8f9ea68226ac98a7', 'request_id': 'req-b700e541-f5a7-463a-a2f8-20b02d4e64fd'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:51.756999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-b700e541-f5a7-463a-a2f8-20b02d4e64fd admin admin] The token cache is being invalidate because project 573da2ba76cb42de8f9ea68226ac98a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:08:51.757359 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-b700e541-f5a7-463a-a2f8-20b02d4e64fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:51.758077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-b700e541-f5a7-463a-a2f8-20b02d4e64fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:51.759618 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1239/2505] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:08:51 2022] DELETE /identity/v3/projects/573da2ba76cb42de8f9ea68226ac98a7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:51.763558 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57201888-ee09-4f74-8ca8-65979f371996 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:51.764976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-57201888-ee09-4f74-8ca8-65979f371996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:51.778044 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57201888-ee09-4f74-8ca8-65979f371996 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:51.835248 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f6419-b451-477d-9581-bbf768653b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:51.835884 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f6419-b451-477d-9581-bbf768653b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:51.836165 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f6419-b451-477d-9581-bbf768653b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:51.836473 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f6419-b451-477d-9581-bbf768653b5b admin admin] PATH_INFO: `/v3/users/eb68525e5d604694b88b7fa2f86de2cb` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:51.837628 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f6419-b451-477d-9581-bbf768653b5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb68525e5d604694b88b7fa2f86de2cb)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:51.839554 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f6419-b451-477d-9581-bbf768653b5b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:51.887428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ec0f6419-b451-477d-9581-bbf768653b5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb68525e5d604694b88b7fa2f86de2cb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:51.899199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ec0f6419-b451-477d-9581-bbf768653b5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb68525e5d604694b88b7fa2f86de2cb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:51.899863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ec0f6419-b451-477d-9581-bbf768653b5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb68525e5d604694b88b7fa2f86de2cb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:51.900504 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ec0f6419-b451-477d-9581-bbf768653b5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb68525e5d604694b88b7fa2f86de2cb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:51.909254 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ec0f6419-b451-477d-9581-bbf768653b5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb68525e5d604694b88b7fa2f86de2cb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:51.924104 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1267/2506] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:08:51 2022] DELETE /identity/v3/users/eb68525e5d604694b88b7fa2f86de2cb => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:08:52.198245 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700e541-f5a7-463a-a2f8-20b02d4e64fd admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:08:52.199710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b700e541-f5a7-463a-a2f8-20b02d4e64fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:08:52.213854 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700e541-f5a7-463a-a2f8-20b02d4e64fd admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:08:52.273428 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20ba19-6689-4f14-bcf3-78db86412b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:08:52.274004 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20ba19-6689-4f14-bcf3-78db86412b0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:08:52.274282 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20ba19-6689-4f14-bcf3-78db86412b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:08:52.274557 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20ba19-6689-4f14-bcf3-78db86412b0b admin admin] PATH_INFO: `/v3/projects/80a66ad9dd584d14be9d285b8a63d25a` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:08:52.275710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20ba19-6689-4f14-bcf3-78db86412b0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=80a66ad9dd584d14be9d285b8a63d25a)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:08:52.277662 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed20ba19-6689-4f14-bcf3-78db86412b0b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:08:52.322157 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ed20ba19-6689-4f14-bcf3-78db86412b0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80a66ad9dd584d14be9d285b8a63d25a', 'request_id': 'req-ed20ba19-6689-4f14-bcf3-78db86412b0b'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:52.327539 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ed20ba19-6689-4f14-bcf3-78db86412b0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80a66ad9dd584d14be9d285b8a63d25a', 'request_id': 'req-ed20ba19-6689-4f14-bcf3-78db86412b0b'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:52.335268 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ed20ba19-6689-4f14-bcf3-78db86412b0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80a66ad9dd584d14be9d285b8a63d25a', 'request_id': 'req-ed20ba19-6689-4f14-bcf3-78db86412b0b'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:52.346784 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ed20ba19-6689-4f14-bcf3-78db86412b0b admin admin] The token cache is being invalidate because project 80a66ad9dd584d14be9d285b8a63d25a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:08:52.347073 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ed20ba19-6689-4f14-bcf3-78db86412b0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:52.347713 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ed20ba19-6689-4f14-bcf3-78db86412b0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:08:52.349205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1240/2507] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:08:52 2022] DELETE /identity/v3/projects/80a66ad9dd584d14be9d285b8a63d25a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:01.919415 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=15,keystone:SELECT=67,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:02.343792 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:DELETE=11,keystone:SELECT=58,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:13.058125 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f6419-b451-477d-9581-bbf768653b5b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:13.059802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ec0f6419-b451-477d-9581-bbf768653b5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:13.134208 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f6419-b451-477d-9581-bbf768653b5b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:13.195646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b92a9-4fd9-4c44-a771-4e9573e9d202 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:13.196261 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b92a9-4fd9-4c44-a771-4e9573e9d202 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:13.196578 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b92a9-4fd9-4c44-a771-4e9573e9d202 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:13.196854 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b92a9-4fd9-4c44-a771-4e9573e9d202 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:13.198624 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b54b92a9-4fd9-4c44-a771-4e9573e9d202 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:13.267091 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b92a9-4fd9-4c44-a771-4e9573e9d202 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:13.268740 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b92a9-4fd9-4c44-a771-4e9573e9d202 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:13.277728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1268/2508] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:09:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:09:20.463993 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20ba19-6689-4f14-bcf3-78db86412b0b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:20.465429 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ed20ba19-6689-4f14-bcf3-78db86412b0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:20.522012 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20ba19-6689-4f14-bcf3-78db86412b0b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:20.526247 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a69f99-b0fe-4f44-a36a-9222a5d782b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:20.526855 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a69f99-b0fe-4f44-a36a-9222a5d782b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:20.527145 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a69f99-b0fe-4f44-a36a-9222a5d782b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:20.527415 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a69f99-b0fe-4f44-a36a-9222a5d782b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:20.529186 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-06a69f99-b0fe-4f44-a36a-9222a5d782b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:20.590041 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a69f99-b0fe-4f44-a36a-9222a5d782b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:20.591463 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a69f99-b0fe-4f44-a36a-9222a5d782b4 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:20.601458 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1241/2509] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:09:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:09:23.273542 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:23.903114 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b92a9-4fd9-4c44-a771-4e9573e9d202 service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:23.904795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b54b92a9-4fd9-4c44-a771-4e9573e9d202 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:23.915684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a69f99-b0fe-4f44-a36a-9222a5d782b4 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:23.917415 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-06a69f99-b0fe-4f44-a36a-9222a5d782b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:23.936863 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a69f99-b0fe-4f44-a36a-9222a5d782b4 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:23.943084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fbaf5-7b79-4658-b660-e9c6613f4ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:23.943947 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fbaf5-7b79-4658-b660-e9c6613f4ede admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:23.944343 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fbaf5-7b79-4658-b660-e9c6613f4ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:23.944908 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fbaf5-7b79-4658-b660-e9c6613f4ede admin admin] PATH_INFO: `/v3/users/257ddc018c33406aa47e7e7129943ae3` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:23.961843 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8fbaf5-7b79-4658-b660-e9c6613f4ede admin admin] RBAC: Authorizing `identity:delete_user(user_id=257ddc018c33406aa47e7e7129943ae3)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:23.963794 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8fbaf5-7b79-4658-b660-e9c6613f4ede admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:24.054066 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b92a9-4fd9-4c44-a771-4e9573e9d202 service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:24.062929 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02cae3-9069-48a6-a3b0-d8c5dc711811 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2244655ba304b9c980b5e68772eefc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:24.063414 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ef8fbaf5-7b79-4658-b660-e9c6613f4ede admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '257ddc018c33406aa47e7e7129943ae3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:24.064015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02cae3-9069-48a6-a3b0-d8c5dc711811 service placement] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:24.064411 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02cae3-9069-48a6-a3b0-d8c5dc711811 service placement] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:24.064758 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02cae3-9069-48a6-a3b0-d8c5dc711811 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:24.066870 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fc02cae3-9069-48a6-a3b0-d8c5dc711811 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:24.073144 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ef8fbaf5-7b79-4658-b660-e9c6613f4ede admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '257ddc018c33406aa47e7e7129943ae3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:24.089427 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc02cae3-9069-48a6-a3b0-d8c5dc711811 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:24.091016 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc02cae3-9069-48a6-a3b0-d8c5dc711811 service placement] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:24.091433 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ef8fbaf5-7b79-4658-b660-e9c6613f4ede admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '257ddc018c33406aa47e7e7129943ae3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:24.092164 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ef8fbaf5-7b79-4658-b660-e9c6613f4ede admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '257ddc018c33406aa47e7e7129943ae3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:24.092804 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ef8fbaf5-7b79-4658-b660-e9c6613f4ede admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '257ddc018c33406aa47e7e7129943ae3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:24.105751 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1269/2510] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:09:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 13:09:24.110511 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1242/2511] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:09:23 2022] DELETE /identity/v3/users/257ddc018c33406aa47e7e7129943ae3 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:09:24.309374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02cae3-9069-48a6-a3b0-d8c5dc711811 service placement] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:24.320729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02cae3-9069-48a6-a3b0-d8c5dc711811 service placement] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:24.386295 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4750d2-f33f-4b4a-9d19-03a3ba1e2232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:24.386864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4750d2-f33f-4b4a-9d19-03a3ba1e2232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:24.387145 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4750d2-f33f-4b4a-9d19-03a3ba1e2232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:24.387413 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4750d2-f33f-4b4a-9d19-03a3ba1e2232 admin admin] PATH_INFO: `/v3/projects/9ce267dd4f15447fae8aef5e99cbb018` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:24.388607 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4750d2-f33f-4b4a-9d19-03a3ba1e2232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ce267dd4f15447fae8aef5e99cbb018)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:24.391174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4750d2-f33f-4b4a-9d19-03a3ba1e2232 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:24.439459 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bb4750d2-f33f-4b4a-9d19-03a3ba1e2232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ce267dd4f15447fae8aef5e99cbb018', 'request_id': 'req-bb4750d2-f33f-4b4a-9d19-03a3ba1e2232'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:24.444852 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bb4750d2-f33f-4b4a-9d19-03a3ba1e2232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ce267dd4f15447fae8aef5e99cbb018', 'request_id': 'req-bb4750d2-f33f-4b4a-9d19-03a3ba1e2232'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:24.454076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bb4750d2-f33f-4b4a-9d19-03a3ba1e2232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ce267dd4f15447fae8aef5e99cbb018', 'request_id': 'req-bb4750d2-f33f-4b4a-9d19-03a3ba1e2232'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:24.465384 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bb4750d2-f33f-4b4a-9d19-03a3ba1e2232 admin admin] The token cache is being invalidate because project 9ce267dd4f15447fae8aef5e99cbb018 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:09:24.465674 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bb4750d2-f33f-4b4a-9d19-03a3ba1e2232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:24.466324 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bb4750d2-f33f-4b4a-9d19-03a3ba1e2232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:24.468044 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1270/2512] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:09:24 2022] DELETE /identity/v3/projects/9ce267dd4f15447fae8aef5e99cbb018 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:24.472858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd34128-5834-43b6-b2b1-02f04b130c5a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:24.473149 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd34128-5834-43b6-b2b1-02f04b130c5a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:24.473424 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd34128-5834-43b6-b2b1-02f04b130c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:24.493391 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-6fd34128-5834-43b6-b2b1-02f04b130c5a None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:24.494589 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6fd34128-5834-43b6-b2b1-02f04b130c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:24.500752 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:24.554845 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1243/2513] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:24.615895 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75139dd0-38e8-436f-9ddf-2a52b0beff90 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:24.616189 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75139dd0-38e8-436f-9ddf-2a52b0beff90 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:24.616501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75139dd0-38e8-436f-9ddf-2a52b0beff90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:24.636780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-75139dd0-38e8-436f-9ddf-2a52b0beff90 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:24.637986 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-75139dd0-38e8-436f-9ddf-2a52b0beff90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:24.645961 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1271/2514] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:24.650100 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd34128-5834-43b6-b2b1-02f04b130c5a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:24.659871 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd34128-5834-43b6-b2b1-02f04b130c5a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:24.664480 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a79b7-503b-48c6-88b2-e539eb529195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:24.665085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a79b7-503b-48c6-88b2-e539eb529195 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:24.665368 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a79b7-503b-48c6-88b2-e539eb529195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:24.665643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a79b7-503b-48c6-88b2-e539eb529195 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:24.666331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a79b7-503b-48c6-88b2-e539eb529195 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:09:24.666637 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a79b7-503b-48c6-88b2-e539eb529195 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:24.668229 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a79b7-503b-48c6-88b2-e539eb529195 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:24.680500 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1244/2515] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 13:09:24.684730 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75139dd0-38e8-436f-9ddf-2a52b0beff90 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:24.694656 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75139dd0-38e8-436f-9ddf-2a52b0beff90 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:24.699237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a20141-459c-4b3f-be9d-45f2a565f16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:24.699863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a20141-459c-4b3f-be9d-45f2a565f16d admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:24.700145 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a20141-459c-4b3f-be9d-45f2a565f16d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:24.700449 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a20141-459c-4b3f-be9d-45f2a565f16d admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:24.701106 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a20141-459c-4b3f-be9d-45f2a565f16d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:24.702989 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a20141-459c-4b3f-be9d-45f2a565f16d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:24.714129 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:24.731890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1272/2516] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:09:24 2022] POST /identity/v3/projects => generated 357 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:09:24.736391 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a79b7-503b-48c6-88b2-e539eb529195 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:24.746167 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a79b7-503b-48c6-88b2-e539eb529195 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:24.799992 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f5d23-b205-4904-b820-645e7095968f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:24.800643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f5d23-b205-4904-b820-645e7095968f admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:24.800941 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f5d23-b205-4904-b820-645e7095968f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:24.801217 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f5d23-b205-4904-b820-645e7095968f admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:24.801876 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f5d23-b205-4904-b820-645e7095968f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:24.803766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f5d23-b205-4904-b820-645e7095968f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:24.824451 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1245/2517] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:09:24 2022] POST /identity/v3/users => generated 279 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:09:24.828716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a20141-459c-4b3f-be9d-45f2a565f16d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:24.838847 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a20141-459c-4b3f-be9d-45f2a565f16d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:24.844012 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa1c9a-d5fc-4dfb-b9f9-964ea37b8323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:24.844654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa1c9a-d5fc-4dfb-b9f9-964ea37b8323 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:24.844934 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa1c9a-d5fc-4dfb-b9f9-964ea37b8323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:24.845208 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa1c9a-d5fc-4dfb-b9f9-964ea37b8323 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:24.845894 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa1c9a-d5fc-4dfb-b9f9-964ea37b8323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:24.847472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa1c9a-d5fc-4dfb-b9f9-964ea37b8323 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:24.857576 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1273/2518] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:09:24.861928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f5d23-b205-4904-b820-645e7095968f admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:24.871729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f5d23-b205-4904-b820-645e7095968f admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:24.876240 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182d0a5-b82b-4f16-832c-2a5c98e8acfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:24.876931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182d0a5-b82b-4f16-832c-2a5c98e8acfe admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:24.877209 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182d0a5-b82b-4f16-832c-2a5c98e8acfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:24.877478 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182d0a5-b82b-4f16-832c-2a5c98e8acfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:24.878154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d182d0a5-b82b-4f16-832c-2a5c98e8acfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:24.879725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d182d0a5-b82b-4f16-832c-2a5c98e8acfe admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:24.889917 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1246/2519] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 13:09:24.893747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa1c9a-d5fc-4dfb-b9f9-964ea37b8323 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:24.903925 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa1c9a-d5fc-4dfb-b9f9-964ea37b8323 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:24.909102 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0912bff-44d9-416c-b871-4921f101772a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:24.909817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0912bff-44d9-416c-b871-4921f101772a admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:24.910203 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0912bff-44d9-416c-b871-4921f101772a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:24.910385 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0912bff-44d9-416c-b871-4921f101772a admin admin] PATH_INFO: `/v3/projects/f11c9e2bbbd9430e82b54354e114ac94/users/1caaca5f82c045898c31c00449768c40/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:24.926554 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0912bff-44d9-416c-b871-4921f101772a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f11c9e2bbbd9430e82b54354e114ac94, user_id=1caaca5f82c045898c31c00449768c40, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:24.929232 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0912bff-44d9-416c-b871-4921f101772a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:24.943867 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1274/2520] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:09:24 2022] PUT /identity/v3/projects/f11c9e2bbbd9430e82b54354e114ac94/users/1caaca5f82c045898c31c00449768c40/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:24.953949 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826d588-4133-4394-bf9b-204e331deaef None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:24.954451 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826d588-4133-4394-bf9b-204e331deaef None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:24.954451 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826d588-4133-4394-bf9b-204e331deaef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:24.982218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c826d588-4133-4394-bf9b-204e331deaef None None] MFA Rules not processed for user `1caaca5f82c045898c31c00449768c40`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:24.983430 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c826d588-4133-4394-bf9b-204e331deaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:25.055649 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1247/2521] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:24 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:25.060572 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0912bff-44d9-416c-b871-4921f101772a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:25.122778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0912bff-44d9-416c-b871-4921f101772a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:25.127407 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdea2d2-fdd2-411e-b048-002ad948b8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:25.128009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdea2d2-fdd2-411e-b048-002ad948b8e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:25.128286 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdea2d2-fdd2-411e-b048-002ad948b8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:25.128586 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdea2d2-fdd2-411e-b048-002ad948b8e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:25.129233 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdea2d2-fdd2-411e-b048-002ad948b8e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:25.131096 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdea2d2-fdd2-411e-b048-002ad948b8e2 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:25.156700 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1275/2522] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:09:25 2022] POST /identity/v3/projects => generated 357 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:09:25.160918 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826d588-4133-4394-bf9b-204e331deaef None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:25.171640 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826d588-4133-4394-bf9b-204e331deaef None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:25.175810 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:25.229357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46caca83-88ad-429f-8b46-161ba825ed5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:25.229900 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46caca83-88ad-429f-8b46-161ba825ed5b admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:25.230180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46caca83-88ad-429f-8b46-161ba825ed5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:25.230453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46caca83-88ad-429f-8b46-161ba825ed5b admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:25.231121 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46caca83-88ad-429f-8b46-161ba825ed5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:25.233071 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46caca83-88ad-429f-8b46-161ba825ed5b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:25.253910 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1248/2523] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:09:25 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:09:25.258001 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdea2d2-fdd2-411e-b048-002ad948b8e2 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:25.268596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdea2d2-fdd2-411e-b048-002ad948b8e2 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:25.273144 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb8346-e8af-4faa-8b9e-4f1eb580a8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:25.273742 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb8346-e8af-4faa-8b9e-4f1eb580a8d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:25.274024 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb8346-e8af-4faa-8b9e-4f1eb580a8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:25.274299 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb8346-e8af-4faa-8b9e-4f1eb580a8d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:25.275014 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb8346-e8af-4faa-8b9e-4f1eb580a8d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:25.276613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb8346-e8af-4faa-8b9e-4f1eb580a8d2 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:25.287715 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1276/2524] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 13:09:25.291724 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46caca83-88ad-429f-8b46-161ba825ed5b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:25.301468 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46caca83-88ad-429f-8b46-161ba825ed5b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:25.306058 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0599c9c-b022-4e88-b9c3-b6e5055815f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:25.306705 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0599c9c-b022-4e88-b9c3-b6e5055815f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:25.306984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0599c9c-b022-4e88-b9c3-b6e5055815f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:25.307259 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0599c9c-b022-4e88-b9c3-b6e5055815f2 admin admin] PATH_INFO: `/v3/projects/085487091d314139bb907fb39acb1900/users/f19a1a27f9b84689b2bacf1902b7fa42/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:25.322599 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0599c9c-b022-4e88-b9c3-b6e5055815f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=085487091d314139bb907fb39acb1900, user_id=f19a1a27f9b84689b2bacf1902b7fa42, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:25.325284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0599c9c-b022-4e88-b9c3-b6e5055815f2 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:25.338350 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1249/2525] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:09:25 2022] PUT /identity/v3/projects/085487091d314139bb907fb39acb1900/users/f19a1a27f9b84689b2bacf1902b7fa42/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:25.342163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb8346-e8af-4faa-8b9e-4f1eb580a8d2 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:25.352955 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb8346-e8af-4faa-8b9e-4f1eb580a8d2 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:25.408358 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8eb441-5122-4bf6-8383-59dc528618ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:25.408982 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8eb441-5122-4bf6-8383-59dc528618ee admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:25.409268 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8eb441-5122-4bf6-8383-59dc528618ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:25.409545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8eb441-5122-4bf6-8383-59dc528618ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:25.410271 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8eb441-5122-4bf6-8383-59dc528618ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:25.411890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8eb441-5122-4bf6-8383-59dc528618ee admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:25.421702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1277/2526] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:09:25.425829 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0599c9c-b022-4e88-b9c3-b6e5055815f2 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:25.435982 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0599c9c-b022-4e88-b9c3-b6e5055815f2 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:25.440863 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ab4e4-2459-4ea7-9c61-67dbfec5c629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:25.441520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ab4e4-2459-4ea7-9c61-67dbfec5c629 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:25.441796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ab4e4-2459-4ea7-9c61-67dbfec5c629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:25.442065 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ab4e4-2459-4ea7-9c61-67dbfec5c629 admin admin] PATH_INFO: `/v3/projects/085487091d314139bb907fb39acb1900/users/f19a1a27f9b84689b2bacf1902b7fa42/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:25.444001 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ab4e4-2459-4ea7-9c61-67dbfec5c629 admin admin] RBAC: Authorizing `identity:create_grant(project_id=085487091d314139bb907fb39acb1900, user_id=f19a1a27f9b84689b2bacf1902b7fa42, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:25.446614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ab4e4-2459-4ea7-9c61-67dbfec5c629 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:25.457447 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1250/2527] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:09:25 2022] PUT /identity/v3/projects/085487091d314139bb907fb39acb1900/users/f19a1a27f9b84689b2bacf1902b7fa42/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:25.467370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130077f6-3995-4525-99f2-fac82434b55d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:25.467769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130077f6-3995-4525-99f2-fac82434b55d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:25.468053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130077f6-3995-4525-99f2-fac82434b55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:25.497544 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-130077f6-3995-4525-99f2-fac82434b55d None None] MFA Rules not processed for user `f19a1a27f9b84689b2bacf1902b7fa42`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:25.498698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-130077f6-3995-4525-99f2-fac82434b55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:25.512819 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:25.580334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1278/2528] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:25 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:25.591527 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ab4e4-2459-4ea7-9c61-67dbfec5c629 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:25.593300 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-002ab4e4-2459-4ea7-9c61-67dbfec5c629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:25.650098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ab4e4-2459-4ea7-9c61-67dbfec5c629 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:25.653719 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55827feb-95c4-434a-964d-9e9e278c3aee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:25.654314 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55827feb-95c4-434a-964d-9e9e278c3aee service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:25.654590 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55827feb-95c4-434a-964d-9e9e278c3aee service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:25.654857 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55827feb-95c4-434a-964d-9e9e278c3aee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:25.663580 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55827feb-95c4-434a-964d-9e9e278c3aee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:25.665018 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55827feb-95c4-434a-964d-9e9e278c3aee service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:25.674146 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1251/2529] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:09:25 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:09:25.746901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130077f6-3995-4525-99f2-fac82434b55d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:25.756886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130077f6-3995-4525-99f2-fac82434b55d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:25.760445 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa73314-95b3-473f-97f6-14728a45d6e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:25.761051 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa73314-95b3-473f-97f6-14728a45d6e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:25.761335 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa73314-95b3-473f-97f6-14728a45d6e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:25.761609 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa73314-95b3-473f-97f6-14728a45d6e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:25.817797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa73314-95b3-473f-97f6-14728a45d6e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:25.819232 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa73314-95b3-473f-97f6-14728a45d6e7 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:25.829142 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1279/2530] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:09:25 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:09:32.335520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55827feb-95c4-434a-964d-9e9e278c3aee service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:32.337107 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-55827feb-95c4-434a-964d-9e9e278c3aee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:32.407118 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55827feb-95c4-434a-964d-9e9e278c3aee service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:32.410720 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f9725-3074-433d-b681-99753515a9c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:32.411331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f9725-3074-433d-b681-99753515a9c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:32.411614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f9725-3074-433d-b681-99753515a9c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:32.411886 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f9725-3074-433d-b681-99753515a9c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:32.413647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2f8f9725-3074-433d-b681-99753515a9c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:32.426887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:32.478739 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f9725-3074-433d-b681-99753515a9c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:32.480172 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f9725-3074-433d-b681-99753515a9c1 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:32.489963 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1252/2531] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:09:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:09:35.824248 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=50 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:35.831805 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa73314-95b3-473f-97f6-14728a45d6e7 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:35.833925 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1fa73314-95b3-473f-97f6-14728a45d6e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:35.854330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa73314-95b3-473f-97f6-14728a45d6e7 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:35.868583 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd40fa-8e3a-4216-80da-fdb50c21243b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:35.869296 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd40fa-8e3a-4216-80da-fdb50c21243b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:35.869576 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd40fa-8e3a-4216-80da-fdb50c21243b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:35.869851 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd40fa-8e3a-4216-80da-fdb50c21243b admin admin] PATH_INFO: `/v3/users/1125a0b93c9d45ccb2d3f28f188dfd04` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:35.871040 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd40fa-8e3a-4216-80da-fdb50c21243b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1125a0b93c9d45ccb2d3f28f188dfd04)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:35.873013 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd40fa-8e3a-4216-80da-fdb50c21243b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:35.925617 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d0fd40fa-8e3a-4216-80da-fdb50c21243b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1125a0b93c9d45ccb2d3f28f188dfd04'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:35.940534 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d0fd40fa-8e3a-4216-80da-fdb50c21243b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1125a0b93c9d45ccb2d3f28f188dfd04'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:35.941232 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d0fd40fa-8e3a-4216-80da-fdb50c21243b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1125a0b93c9d45ccb2d3f28f188dfd04'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:35.941841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d0fd40fa-8e3a-4216-80da-fdb50c21243b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1125a0b93c9d45ccb2d3f28f188dfd04'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:35.950019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d0fd40fa-8e3a-4216-80da-fdb50c21243b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1125a0b93c9d45ccb2d3f28f188dfd04'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:35.963505 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1280/2532] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:09:35 2022] DELETE /identity/v3/users/1125a0b93c9d45ccb2d3f28f188dfd04 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:36.135258 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f9725-3074-433d-b681-99753515a9c1 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:36.150440 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f9725-3074-433d-b681-99753515a9c1 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:36.207029 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a2374-d9cc-4825-b988-cf74a026f134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:36.207667 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a2374-d9cc-4825-b988-cf74a026f134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:36.207944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a2374-d9cc-4825-b988-cf74a026f134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:36.208212 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a2374-d9cc-4825-b988-cf74a026f134 admin admin] PATH_INFO: `/v3/projects/7a6de534dbd541828ac211fae35b0ce4` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:36.209429 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a2374-d9cc-4825-b988-cf74a026f134 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a6de534dbd541828ac211fae35b0ce4)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:36.211345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a2374-d9cc-4825-b988-cf74a026f134 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:36.256468 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-138a2374-d9cc-4825-b988-cf74a026f134 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a6de534dbd541828ac211fae35b0ce4', 'request_id': 'req-138a2374-d9cc-4825-b988-cf74a026f134'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:36.261912 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-138a2374-d9cc-4825-b988-cf74a026f134 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a6de534dbd541828ac211fae35b0ce4', 'request_id': 'req-138a2374-d9cc-4825-b988-cf74a026f134'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:36.269231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-138a2374-d9cc-4825-b988-cf74a026f134 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a6de534dbd541828ac211fae35b0ce4', 'request_id': 'req-138a2374-d9cc-4825-b988-cf74a026f134'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:36.279894 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-138a2374-d9cc-4825-b988-cf74a026f134 admin admin] The token cache is being invalidate because project 7a6de534dbd541828ac211fae35b0ce4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:09:36.280175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-138a2374-d9cc-4825-b988-cf74a026f134 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:36.281316 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-138a2374-d9cc-4825-b988-cf74a026f134 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:36.282773 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1253/2533] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:09:36 2022] DELETE /identity/v3/projects/7a6de534dbd541828ac211fae35b0ce4 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:36.287587 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c7dcd-71b3-446d-b698-215a4baafb32 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:36.287861 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c7dcd-71b3-446d-b698-215a4baafb32 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:36.288148 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c7dcd-71b3-446d-b698-215a4baafb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:36.308215 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e01c7dcd-71b3-446d-b698-215a4baafb32 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:36.309356 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e01c7dcd-71b3-446d-b698-215a4baafb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:36.367348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1281/2534] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:36.374959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434febf-3d15-4139-ac75-09f8f3d457ac None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:36.375269 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434febf-3d15-4139-ac75-09f8f3d457ac None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:36.375543 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434febf-3d15-4139-ac75-09f8f3d457ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:36.394533 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-0434febf-3d15-4139-ac75-09f8f3d457ac None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:36.395710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0434febf-3d15-4139-ac75-09f8f3d457ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:36.403390 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1254/2535] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:36.407825 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c7dcd-71b3-446d-b698-215a4baafb32 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:36.418066 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c7dcd-71b3-446d-b698-215a4baafb32 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:36.423147 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd3d43-e0ae-4c0a-af5a-9516d414872f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:36.423747 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd3d43-e0ae-4c0a-af5a-9516d414872f admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:36.424021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd3d43-e0ae-4c0a-af5a-9516d414872f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:36.424289 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd3d43-e0ae-4c0a-af5a-9516d414872f admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:36.425003 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd3d43-e0ae-4c0a-af5a-9516d414872f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:09:36.425300 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd3d43-e0ae-4c0a-af5a-9516d414872f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:36.426869 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd3d43-e0ae-4c0a-af5a-9516d414872f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:36.438132 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1282/2536] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 13:09:36.442264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434febf-3d15-4139-ac75-09f8f3d457ac None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:36.451334 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434febf-3d15-4139-ac75-09f8f3d457ac None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:36.456466 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d374bd-ab94-4531-82d5-b5eeae234e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:36.457062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d374bd-ab94-4531-82d5-b5eeae234e56 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:36.457336 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d374bd-ab94-4531-82d5-b5eeae234e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:36.457604 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d374bd-ab94-4531-82d5-b5eeae234e56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:36.458726 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d374bd-ab94-4531-82d5-b5eeae234e56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:36.460253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d374bd-ab94-4531-82d5-b5eeae234e56 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:36.489917 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1255/2537] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:09:36 2022] POST /identity/v3/projects => generated 403 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:09:36.494121 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd3d43-e0ae-4c0a-af5a-9516d414872f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:36.504239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd3d43-e0ae-4c0a-af5a-9516d414872f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:36.559133 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec18c2c-f1f8-4609-9595-f9ea1b400ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:36.559674 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec18c2c-f1f8-4609-9595-f9ea1b400ae1 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:36.559957 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec18c2c-f1f8-4609-9595-f9ea1b400ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:36.560235 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec18c2c-f1f8-4609-9595-f9ea1b400ae1 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:36.560928 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec18c2c-f1f8-4609-9595-f9ea1b400ae1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:36.562851 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec18c2c-f1f8-4609-9595-f9ea1b400ae1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:36.570640 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:36.585663 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1283/2538] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:09:36 2022] POST /identity/v3/users => generated 302 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:09:36.589890 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d374bd-ab94-4531-82d5-b5eeae234e56 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:36.602626 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d374bd-ab94-4531-82d5-b5eeae234e56 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:36.607581 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b62c29-4a39-4941-815e-852ea2edfb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:36.608192 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b62c29-4a39-4941-815e-852ea2edfb1e admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:36.608562 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b62c29-4a39-4941-815e-852ea2edfb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:36.608845 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b62c29-4a39-4941-815e-852ea2edfb1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:36.609576 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b62c29-4a39-4941-815e-852ea2edfb1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:36.611199 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b62c29-4a39-4941-815e-852ea2edfb1e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:36.621176 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1256/2539] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:09:36.625194 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec18c2c-f1f8-4609-9595-f9ea1b400ae1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:36.635275 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec18c2c-f1f8-4609-9595-f9ea1b400ae1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:36.639848 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a2b65-2503-4084-b09e-011b5fbaa27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:36.640982 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a2b65-2503-4084-b09e-011b5fbaa27f admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:36.641261 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a2b65-2503-4084-b09e-011b5fbaa27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:36.641725 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a2b65-2503-4084-b09e-011b5fbaa27f admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:36.642419 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a2b65-2503-4084-b09e-011b5fbaa27f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:36.644070 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a2b65-2503-4084-b09e-011b5fbaa27f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:36.654438 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1284/2540] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:09:36.658961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b62c29-4a39-4941-815e-852ea2edfb1e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:36.664114 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:36.670545 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b62c29-4a39-4941-815e-852ea2edfb1e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:36.676030 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef13cb-81d5-493e-9622-98f82a005db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:36.676713 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef13cb-81d5-493e-9622-98f82a005db2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:36.676996 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef13cb-81d5-493e-9622-98f82a005db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:36.677272 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef13cb-81d5-493e-9622-98f82a005db2 admin admin] PATH_INFO: `/v3/projects/fc6207bdb34446c398ba22daf0147a21/users/d66d14a8857e49a199a48c0ec043487c/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:36.691589 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef13cb-81d5-493e-9622-98f82a005db2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc6207bdb34446c398ba22daf0147a21, user_id=d66d14a8857e49a199a48c0ec043487c, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:36.694272 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef13cb-81d5-493e-9622-98f82a005db2 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:36.707563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1257/2541] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:09:36 2022] PUT /identity/v3/projects/fc6207bdb34446c398ba22daf0147a21/users/d66d14a8857e49a199a48c0ec043487c/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:36.717410 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd037782-f20e-42cd-848a-c792c0195514 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:36.717691 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd037782-f20e-42cd-848a-c792c0195514 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:36.717960 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd037782-f20e-42cd-848a-c792c0195514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:36.746315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-bd037782-f20e-42cd-848a-c792c0195514 None None] MFA Rules not processed for user `d66d14a8857e49a199a48c0ec043487c`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:36.747547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bd037782-f20e-42cd-848a-c792c0195514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:36.820925 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1285/2542] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:36 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:36.826121 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef13cb-81d5-493e-9622-98f82a005db2 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:36.894221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef13cb-81d5-493e-9622-98f82a005db2 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:36.899869 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428e086-441a-4263-ae95-17d510f53d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:36.900522 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428e086-441a-4263-ae95-17d510f53d90 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:36.900811 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428e086-441a-4263-ae95-17d510f53d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:36.901086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428e086-441a-4263-ae95-17d510f53d90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:36.901758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2428e086-441a-4263-ae95-17d510f53d90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:36.903670 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2428e086-441a-4263-ae95-17d510f53d90 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:36.930512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1258/2543] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:09:36 2022] POST /identity/v3/projects => generated 405 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:09:36.934610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd037782-f20e-42cd-848a-c792c0195514 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:36.947010 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd037782-f20e-42cd-848a-c792c0195514 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:37.004641 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9ac76-a4fe-47b2-843f-e79cd6598a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:37.005269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9ac76-a4fe-47b2-843f-e79cd6598a6e admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:37.005473 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9ac76-a4fe-47b2-843f-e79cd6598a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:37.005751 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9ac76-a4fe-47b2-843f-e79cd6598a6e admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:37.006428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9ac76-a4fe-47b2-843f-e79cd6598a6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:37.008826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9ac76-a4fe-47b2-843f-e79cd6598a6e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:37.032136 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1286/2544] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:09:36 2022] POST /identity/v3/users => generated 309 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:09:37.036170 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428e086-441a-4263-ae95-17d510f53d90 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:37.046527 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428e086-441a-4263-ae95-17d510f53d90 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:37.051180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc68a1d-9fb5-4799-8ca4-41def56efddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:37.051785 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc68a1d-9fb5-4799-8ca4-41def56efddc admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:37.052066 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc68a1d-9fb5-4799-8ca4-41def56efddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:37.052371 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc68a1d-9fb5-4799-8ca4-41def56efddc admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:37.053063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc68a1d-9fb5-4799-8ca4-41def56efddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:37.054673 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc68a1d-9fb5-4799-8ca4-41def56efddc admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:37.065532 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1259/2545] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 13:09:37.069466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9ac76-a4fe-47b2-843f-e79cd6598a6e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:37.079892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9ac76-a4fe-47b2-843f-e79cd6598a6e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:37.084555 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f39ad0-0254-41ae-8ed9-4c35f40d4c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:37.085207 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f39ad0-0254-41ae-8ed9-4c35f40d4c0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:37.085492 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f39ad0-0254-41ae-8ed9-4c35f40d4c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:37.085780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f39ad0-0254-41ae-8ed9-4c35f40d4c0d admin admin] PATH_INFO: `/v3/projects/471f5dcdecbd4ed9a6aeb3727399d24a/users/d68ff5bb397d497f8f7bf9ecf07d8519/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:37.100543 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f39ad0-0254-41ae-8ed9-4c35f40d4c0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=471f5dcdecbd4ed9a6aeb3727399d24a, user_id=d68ff5bb397d497f8f7bf9ecf07d8519, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:37.103291 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f39ad0-0254-41ae-8ed9-4c35f40d4c0d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:37.115295 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1287/2546] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:09:37 2022] PUT /identity/v3/projects/471f5dcdecbd4ed9a6aeb3727399d24a/users/d68ff5bb397d497f8f7bf9ecf07d8519/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:37.119200 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc68a1d-9fb5-4799-8ca4-41def56efddc admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:37.129840 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc68a1d-9fb5-4799-8ca4-41def56efddc admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:37.188805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239153f-d95e-40f1-a73a-a8bc4ef3c037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:37.189419 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239153f-d95e-40f1-a73a-a8bc4ef3c037 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:37.189700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239153f-d95e-40f1-a73a-a8bc4ef3c037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:37.189979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239153f-d95e-40f1-a73a-a8bc4ef3c037 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:37.190665 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239153f-d95e-40f1-a73a-a8bc4ef3c037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:37.192257 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239153f-d95e-40f1-a73a-a8bc4ef3c037 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:37.201773 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1260/2547] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:09:37.206518 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f39ad0-0254-41ae-8ed9-4c35f40d4c0d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:37.216613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f39ad0-0254-41ae-8ed9-4c35f40d4c0d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:37.221469 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0356f1-2748-4932-81a2-0e9dd0875fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:37.222107 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0356f1-2748-4932-81a2-0e9dd0875fbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:37.222381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0356f1-2748-4932-81a2-0e9dd0875fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:37.222650 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0356f1-2748-4932-81a2-0e9dd0875fbe admin admin] PATH_INFO: `/v3/projects/471f5dcdecbd4ed9a6aeb3727399d24a/users/d68ff5bb397d497f8f7bf9ecf07d8519/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:37.224863 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0356f1-2748-4932-81a2-0e9dd0875fbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=471f5dcdecbd4ed9a6aeb3727399d24a, user_id=d68ff5bb397d497f8f7bf9ecf07d8519, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:37.227465 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0356f1-2748-4932-81a2-0e9dd0875fbe admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:37.238001 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1288/2548] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:09:37 2022] PUT /identity/v3/projects/471f5dcdecbd4ed9a6aeb3727399d24a/users/d68ff5bb397d497f8f7bf9ecf07d8519/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:37.248612 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75d0e2-3d6b-4d59-a40c-03e8ab156b04 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:37.248930 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75d0e2-3d6b-4d59-a40c-03e8ab156b04 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:37.249212 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75d0e2-3d6b-4d59-a40c-03e8ab156b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:37.276729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-ed75d0e2-3d6b-4d59-a40c-03e8ab156b04 None None] MFA Rules not processed for user `d68ff5bb397d497f8f7bf9ecf07d8519`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:37.277932 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ed75d0e2-3d6b-4d59-a40c-03e8ab156b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:37.315531 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:37.345942 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1261/2549] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:37 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:37.356796 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0356f1-2748-4932-81a2-0e9dd0875fbe admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:37.358499 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cc0356f1-2748-4932-81a2-0e9dd0875fbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:37.381358 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:37.416448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0356f1-2748-4932-81a2-0e9dd0875fbe admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:37.419980 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d9e0e-fb39-4b00-bd22-0cf8e2484947 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:37.420613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d9e0e-fb39-4b00-bd22-0cf8e2484947 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:37.420890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d9e0e-fb39-4b00-bd22-0cf8e2484947 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:37.421161 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d9e0e-fb39-4b00-bd22-0cf8e2484947 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:37.430377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d9e0e-fb39-4b00-bd22-0cf8e2484947 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:37.431774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d9e0e-fb39-4b00-bd22-0cf8e2484947 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:37.441550 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1289/2550] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:09:37 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:09:37.587029 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75d0e2-3d6b-4d59-a40c-03e8ab156b04 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:37.647818 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75d0e2-3d6b-4d59-a40c-03e8ab156b04 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:37.652219 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fceb6-0c0b-461a-aa5a-c92ebdce49ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:37.652873 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fceb6-0c0b-461a-aa5a-c92ebdce49ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:37.653160 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fceb6-0c0b-461a-aa5a-c92ebdce49ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:37.653436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34fceb6-0c0b-461a-aa5a-c92ebdce49ec admin admin] PATH_INFO: `/v3/users/d66d14a8857e49a199a48c0ec043487c` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:37.654585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34fceb6-0c0b-461a-aa5a-c92ebdce49ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=d66d14a8857e49a199a48c0ec043487c)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:37.657071 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34fceb6-0c0b-461a-aa5a-c92ebdce49ec admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:37.702456 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a34fceb6-0c0b-461a-aa5a-c92ebdce49ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd66d14a8857e49a199a48c0ec043487c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:37.709190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a34fceb6-0c0b-461a-aa5a-c92ebdce49ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd66d14a8857e49a199a48c0ec043487c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:37.721035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a34fceb6-0c0b-461a-aa5a-c92ebdce49ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd66d14a8857e49a199a48c0ec043487c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:37.721734 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a34fceb6-0c0b-461a-aa5a-c92ebdce49ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd66d14a8857e49a199a48c0ec043487c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:37.722341 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a34fceb6-0c0b-461a-aa5a-c92ebdce49ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd66d14a8857e49a199a48c0ec043487c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:37.735409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1262/2551] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:09:37 2022] DELETE /identity/v3/users/d66d14a8857e49a199a48c0ec043487c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:37.739943 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621d7c6-6607-472b-b4eb-c9a69a7c1059 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:37.740233 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621d7c6-6607-472b-b4eb-c9a69a7c1059 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:37.740548 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621d7c6-6607-472b-b4eb-c9a69a7c1059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:37.758899 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9621d7c6-6607-472b-b4eb-c9a69a7c1059 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:37.760097 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9621d7c6-6607-472b-b4eb-c9a69a7c1059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:37.817095 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1290/2552] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:37.826699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fceb6-0c0b-461a-aa5a-c92ebdce49ec admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:37.828124 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a34fceb6-0c0b-461a-aa5a-c92ebdce49ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:37.885574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34fceb6-0c0b-461a-aa5a-c92ebdce49ec admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:37.889160 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dccb427-4885-4b9b-8a50-c3c8c4d8bd8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:37.889764 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dccb427-4885-4b9b-8a50-c3c8c4d8bd8a service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:37.890036 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dccb427-4885-4b9b-8a50-c3c8c4d8bd8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:37.890301 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dccb427-4885-4b9b-8a50-c3c8c4d8bd8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:37.897799 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:37.900386 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dccb427-4885-4b9b-8a50-c3c8c4d8bd8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:37.902172 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dccb427-4885-4b9b-8a50-c3c8c4d8bd8a service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:37.916208 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1263/2553] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:09:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:09:38.218077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621d7c6-6607-472b-b4eb-c9a69a7c1059 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:38.220078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9621d7c6-6607-472b-b4eb-c9a69a7c1059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:38.232987 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621d7c6-6607-472b-b4eb-c9a69a7c1059 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:38.238323 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a5d6c-0765-4bb0-8844-1b44ef65ed9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:38.238968 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a5d6c-0765-4bb0-8844-1b44ef65ed9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:38.239249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a5d6c-0765-4bb0-8844-1b44ef65ed9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:38.239522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a5d6c-0765-4bb0-8844-1b44ef65ed9e admin admin] PATH_INFO: `/v3/projects/fc6207bdb34446c398ba22daf0147a21` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:38.240716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721a5d6c-0765-4bb0-8844-1b44ef65ed9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc6207bdb34446c398ba22daf0147a21)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:38.242683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721a5d6c-0765-4bb0-8844-1b44ef65ed9e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:38.289743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-721a5d6c-0765-4bb0-8844-1b44ef65ed9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc6207bdb34446c398ba22daf0147a21', 'request_id': 'req-721a5d6c-0765-4bb0-8844-1b44ef65ed9e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:38.295306 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-721a5d6c-0765-4bb0-8844-1b44ef65ed9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc6207bdb34446c398ba22daf0147a21', 'request_id': 'req-721a5d6c-0765-4bb0-8844-1b44ef65ed9e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:38.302687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-721a5d6c-0765-4bb0-8844-1b44ef65ed9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc6207bdb34446c398ba22daf0147a21', 'request_id': 'req-721a5d6c-0765-4bb0-8844-1b44ef65ed9e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:38.314594 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-721a5d6c-0765-4bb0-8844-1b44ef65ed9e admin admin] The token cache is being invalidate because project fc6207bdb34446c398ba22daf0147a21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:09:38.314884 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-721a5d6c-0765-4bb0-8844-1b44ef65ed9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:38.315513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-721a5d6c-0765-4bb0-8844-1b44ef65ed9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:38.317126 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1291/2554] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:09:38 2022] DELETE /identity/v3/projects/fc6207bdb34446c398ba22daf0147a21 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:38.321188 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dccb427-4885-4b9b-8a50-c3c8c4d8bd8a service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:38.331362 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dccb427-4885-4b9b-8a50-c3c8c4d8bd8a service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:38.385687 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33fbb49-db46-46fe-8157-f5af267a6f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:38.386317 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fbb49-db46-46fe-8157-f5af267a6f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:38.386602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fbb49-db46-46fe-8157-f5af267a6f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:38.386877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fbb49-db46-46fe-8157-f5af267a6f60 admin admin] PATH_INFO: `/v3/users/d68ff5bb397d497f8f7bf9ecf07d8519` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:38.388019 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33fbb49-db46-46fe-8157-f5af267a6f60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d68ff5bb397d497f8f7bf9ecf07d8519)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:38.390005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33fbb49-db46-46fe-8157-f5af267a6f60 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:38.434492 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a33fbb49-db46-46fe-8157-f5af267a6f60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd68ff5bb397d497f8f7bf9ecf07d8519'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:38.441742 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a33fbb49-db46-46fe-8157-f5af267a6f60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd68ff5bb397d497f8f7bf9ecf07d8519'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:38.453346 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a33fbb49-db46-46fe-8157-f5af267a6f60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd68ff5bb397d497f8f7bf9ecf07d8519'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:38.454583 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a33fbb49-db46-46fe-8157-f5af267a6f60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd68ff5bb397d497f8f7bf9ecf07d8519'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:38.455204 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-a33fbb49-db46-46fe-8157-f5af267a6f60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd68ff5bb397d497f8f7bf9ecf07d8519'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:38.467612 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1264/2555] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:09:38 2022] DELETE /identity/v3/users/d68ff5bb397d497f8f7bf9ecf07d8519 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:38.742000 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a5d6c-0765-4bb0-8844-1b44ef65ed9e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:38.743501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-721a5d6c-0765-4bb0-8844-1b44ef65ed9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:38.757274 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a5d6c-0765-4bb0-8844-1b44ef65ed9e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:38.800786 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:38.815706 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813e4c5-73c2-41d7-8a98-c09dd3484d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:38.816364 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813e4c5-73c2-41d7-8a98-c09dd3484d0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:38.816647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813e4c5-73c2-41d7-8a98-c09dd3484d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:38.816922 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813e4c5-73c2-41d7-8a98-c09dd3484d0d admin admin] PATH_INFO: `/v3/projects/471f5dcdecbd4ed9a6aeb3727399d24a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:38.818071 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813e4c5-73c2-41d7-8a98-c09dd3484d0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=471f5dcdecbd4ed9a6aeb3727399d24a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:38.819968 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813e4c5-73c2-41d7-8a98-c09dd3484d0d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:38.865434 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6813e4c5-73c2-41d7-8a98-c09dd3484d0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '471f5dcdecbd4ed9a6aeb3727399d24a', 'request_id': 'req-6813e4c5-73c2-41d7-8a98-c09dd3484d0d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:38.870772 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6813e4c5-73c2-41d7-8a98-c09dd3484d0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '471f5dcdecbd4ed9a6aeb3727399d24a', 'request_id': 'req-6813e4c5-73c2-41d7-8a98-c09dd3484d0d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:38.878196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6813e4c5-73c2-41d7-8a98-c09dd3484d0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '471f5dcdecbd4ed9a6aeb3727399d24a', 'request_id': 'req-6813e4c5-73c2-41d7-8a98-c09dd3484d0d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:38.889414 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6813e4c5-73c2-41d7-8a98-c09dd3484d0d admin admin] The token cache is being invalidate because project 471f5dcdecbd4ed9a6aeb3727399d24a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:09:38.889697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6813e4c5-73c2-41d7-8a98-c09dd3484d0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:38.890319 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6813e4c5-73c2-41d7-8a98-c09dd3484d0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:38.891796 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1292/2556] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:09:38 2022] DELETE /identity/v3/projects/471f5dcdecbd4ed9a6aeb3727399d24a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:38.897187 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7ff0f-0654-4691-8564-a2c9986e067d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:38.897475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7ff0f-0654-4691-8564-a2c9986e067d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:38.897749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7ff0f-0654-4691-8564-a2c9986e067d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:38.918448 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-31f7ff0f-0654-4691-8564-a2c9986e067d None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:38.919640 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-31f7ff0f-0654-4691-8564-a2c9986e067d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:38.978725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1265/2557] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:38.986636 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5548943-feb2-423a-9a2f-2d60d40c907a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:38.986921 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5548943-feb2-423a-9a2f-2d60d40c907a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:38.987195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5548943-feb2-423a-9a2f-2d60d40c907a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:39.007418 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f5548943-feb2-423a-9a2f-2d60d40c907a None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:39.008637 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f5548943-feb2-423a-9a2f-2d60d40c907a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:39.016508 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1293/2558] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:39.020755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7ff0f-0654-4691-8564-a2c9986e067d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:39.030222 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7ff0f-0654-4691-8564-a2c9986e067d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:39.034684 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b330f15-dbea-4c7a-b956-d7d838649adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:39.035282 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b330f15-dbea-4c7a-b956-d7d838649adf admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:39.035557 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b330f15-dbea-4c7a-b956-d7d838649adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:39.035824 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b330f15-dbea-4c7a-b956-d7d838649adf admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:39.036534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b330f15-dbea-4c7a-b956-d7d838649adf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:09:39.036831 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b330f15-dbea-4c7a-b956-d7d838649adf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:39.038406 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b330f15-dbea-4c7a-b956-d7d838649adf admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:39.050166 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1266/2559] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 13:09:39.054144 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5548943-feb2-423a-9a2f-2d60d40c907a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:39.063705 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5548943-feb2-423a-9a2f-2d60d40c907a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:39.068273 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b18a5-22f9-4194-bba6-e7c8e59be0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:39.068887 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b18a5-22f9-4194-bba6-e7c8e59be0fd admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:39.069163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b18a5-22f9-4194-bba6-e7c8e59be0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:39.069435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b18a5-22f9-4194-bba6-e7c8e59be0fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:39.070094 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b18a5-22f9-4194-bba6-e7c8e59be0fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:39.071982 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b18a5-22f9-4194-bba6-e7c8e59be0fd admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:39.098304 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1294/2560] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:09:39 2022] POST /identity/v3/projects => generated 399 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:09:39.102547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b330f15-dbea-4c7a-b956-d7d838649adf admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:39.107647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:39.114045 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b330f15-dbea-4c7a-b956-d7d838649adf admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:39.169328 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7f2ca-7a53-4869-b538-e69080d10cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:39.169930 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7f2ca-7a53-4869-b538-e69080d10cdc admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:39.170207 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7f2ca-7a53-4869-b538-e69080d10cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:39.170478 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7f2ca-7a53-4869-b538-e69080d10cdc admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:39.171133 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7f2ca-7a53-4869-b538-e69080d10cdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:39.173614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7f2ca-7a53-4869-b538-e69080d10cdc admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:39.193638 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1267/2561] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:09:39 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:09:39.197763 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b18a5-22f9-4194-bba6-e7c8e59be0fd admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:39.208977 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b18a5-22f9-4194-bba6-e7c8e59be0fd admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:39.213769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba23f7-c1ad-443c-b427-1ade04113e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:39.214376 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba23f7-c1ad-443c-b427-1ade04113e84 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:39.214654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba23f7-c1ad-443c-b427-1ade04113e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:39.214923 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba23f7-c1ad-443c-b427-1ade04113e84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:39.215608 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba23f7-c1ad-443c-b427-1ade04113e84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:39.217231 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba23f7-c1ad-443c-b427-1ade04113e84 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:39.233566 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1295/2562] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 13:09:39.234264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7f2ca-7a53-4869-b538-e69080d10cdc admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:39.246084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7f2ca-7a53-4869-b538-e69080d10cdc admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:39.250965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590a1f9-0c1b-4bd8-9556-317540c53428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:39.251579 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590a1f9-0c1b-4bd8-9556-317540c53428 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:39.251857 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590a1f9-0c1b-4bd8-9556-317540c53428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:39.252128 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590a1f9-0c1b-4bd8-9556-317540c53428 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:39.252871 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590a1f9-0c1b-4bd8-9556-317540c53428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:39.254486 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590a1f9-0c1b-4bd8-9556-317540c53428 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:39.269476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1268/2563] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 13:09:39.271277 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba23f7-c1ad-443c-b427-1ade04113e84 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:39.285211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba23f7-c1ad-443c-b427-1ade04113e84 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:39.290907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893e241-3dc2-4f74-a0ad-7eda7d836322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:39.291563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893e241-3dc2-4f74-a0ad-7eda7d836322 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:39.292010 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893e241-3dc2-4f74-a0ad-7eda7d836322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:39.292284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893e241-3dc2-4f74-a0ad-7eda7d836322 admin admin] PATH_INFO: `/v3/projects/7221efb3614a41748e3cadfc7a9326fd/users/ab7292a8d3554a64818e5cfc4d8efbc0/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:39.309247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893e241-3dc2-4f74-a0ad-7eda7d836322 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7221efb3614a41748e3cadfc7a9326fd, user_id=ab7292a8d3554a64818e5cfc4d8efbc0, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:39.311890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893e241-3dc2-4f74-a0ad-7eda7d836322 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:39.326136 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1296/2564] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:09:39 2022] PUT /identity/v3/projects/7221efb3614a41748e3cadfc7a9326fd/users/ab7292a8d3554a64818e5cfc4d8efbc0/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:39.337406 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b877aa-56cc-4621-a6ac-805e6112d936 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:39.337914 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b877aa-56cc-4621-a6ac-805e6112d936 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:39.338124 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b877aa-56cc-4621-a6ac-805e6112d936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:39.368788 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-d7b877aa-56cc-4621-a6ac-805e6112d936 None None] MFA Rules not processed for user `ab7292a8d3554a64818e5cfc4d8efbc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:39.369921 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d7b877aa-56cc-4621-a6ac-805e6112d936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:39.435394 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1269/2565] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:39 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:39.439855 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893e241-3dc2-4f74-a0ad-7eda7d836322 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:39.502088 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893e241-3dc2-4f74-a0ad-7eda7d836322 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:39.506759 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f380941-3180-4fe9-a959-c30a31004757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:39.507373 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f380941-3180-4fe9-a959-c30a31004757 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:39.507659 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f380941-3180-4fe9-a959-c30a31004757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:39.507937 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f380941-3180-4fe9-a959-c30a31004757 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:39.508640 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f380941-3180-4fe9-a959-c30a31004757 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:39.510519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f380941-3180-4fe9-a959-c30a31004757 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:39.531690 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:39.543019 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1297/2566] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:09:39 2022] POST /identity/v3/projects => generated 401 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:09:39.544325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b877aa-56cc-4621-a6ac-805e6112d936 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:39.555972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b877aa-56cc-4621-a6ac-805e6112d936 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:39.618892 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59af7ec-1170-44db-ab21-235ee6d3b333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:39.619520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59af7ec-1170-44db-ab21-235ee6d3b333 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:39.619804 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59af7ec-1170-44db-ab21-235ee6d3b333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:39.620083 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59af7ec-1170-44db-ab21-235ee6d3b333 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:39.620782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59af7ec-1170-44db-ab21-235ee6d3b333 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:39.622693 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59af7ec-1170-44db-ab21-235ee6d3b333 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:39.648632 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:39.654846 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1270/2567] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:09:39 2022] POST /identity/v3/users => generated 307 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:09:39.659418 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f380941-3180-4fe9-a959-c30a31004757 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:39.669797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f380941-3180-4fe9-a959-c30a31004757 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:39.674450 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d222c-2aaf-4e7e-9a0b-1e5bb99df033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:39.675055 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d222c-2aaf-4e7e-9a0b-1e5bb99df033 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:39.675338 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d222c-2aaf-4e7e-9a0b-1e5bb99df033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:39.675614 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d222c-2aaf-4e7e-9a0b-1e5bb99df033 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:39.676321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d222c-2aaf-4e7e-9a0b-1e5bb99df033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:39.677925 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d222c-2aaf-4e7e-9a0b-1e5bb99df033 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:39.688360 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1298/2568] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:09:39.692524 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59af7ec-1170-44db-ab21-235ee6d3b333 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:39.702400 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59af7ec-1170-44db-ab21-235ee6d3b333 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:39.707470 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc591c7-3bd0-42d5-bfb4-284e6446b76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:39.708132 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc591c7-3bd0-42d5-bfb4-284e6446b76e admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:39.708436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc591c7-3bd0-42d5-bfb4-284e6446b76e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:39.708714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc591c7-3bd0-42d5-bfb4-284e6446b76e admin admin] PATH_INFO: `/v3/projects/eee8a7fcac9b4789b005753fe674afa1/users/a717e20ac4714379ba2f8463d32150f5/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:39.723678 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc591c7-3bd0-42d5-bfb4-284e6446b76e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eee8a7fcac9b4789b005753fe674afa1, user_id=a717e20ac4714379ba2f8463d32150f5, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:39.726731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc591c7-3bd0-42d5-bfb4-284e6446b76e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:39.738748 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1271/2569] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:09:39 2022] PUT /identity/v3/projects/eee8a7fcac9b4789b005753fe674afa1/users/a717e20ac4714379ba2f8463d32150f5/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:39.742567 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d222c-2aaf-4e7e-9a0b-1e5bb99df033 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:39.752738 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d222c-2aaf-4e7e-9a0b-1e5bb99df033 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:39.808121 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb892ce-e383-4d4f-83ca-0bb5286f80bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:39.808803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb892ce-e383-4d4f-83ca-0bb5286f80bb admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:39.809132 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb892ce-e383-4d4f-83ca-0bb5286f80bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:39.809459 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb892ce-e383-4d4f-83ca-0bb5286f80bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:39.810680 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb892ce-e383-4d4f-83ca-0bb5286f80bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:39.811901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb892ce-e383-4d4f-83ca-0bb5286f80bb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:39.822379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1299/2570] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:09:39.826419 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc591c7-3bd0-42d5-bfb4-284e6446b76e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:39.837259 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc591c7-3bd0-42d5-bfb4-284e6446b76e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:39.841773 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d088d79-127f-4474-9ec6-5169ce3318ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:39.842415 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d088d79-127f-4474-9ec6-5169ce3318ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:39.842710 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d088d79-127f-4474-9ec6-5169ce3318ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:39.842980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d088d79-127f-4474-9ec6-5169ce3318ee admin admin] PATH_INFO: `/v3/projects/eee8a7fcac9b4789b005753fe674afa1/users/a717e20ac4714379ba2f8463d32150f5/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:39.844981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d088d79-127f-4474-9ec6-5169ce3318ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=eee8a7fcac9b4789b005753fe674afa1, user_id=a717e20ac4714379ba2f8463d32150f5, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:39.847751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d088d79-127f-4474-9ec6-5169ce3318ee admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:39.858024 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1272/2571] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:09:39 2022] PUT /identity/v3/projects/eee8a7fcac9b4789b005753fe674afa1/users/a717e20ac4714379ba2f8463d32150f5/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:39.867577 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f79ee4-9355-43fb-b824-f0ec70e8dc98 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:39.867872 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f79ee4-9355-43fb-b824-f0ec70e8dc98 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:39.868155 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f79ee4-9355-43fb-b824-f0ec70e8dc98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:39.896786 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b2f79ee4-9355-43fb-b824-f0ec70e8dc98 None None] MFA Rules not processed for user `a717e20ac4714379ba2f8463d32150f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:39.898017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b2f79ee4-9355-43fb-b824-f0ec70e8dc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:39.966211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1300/2572] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:39 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:39.977560 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d088d79-127f-4474-9ec6-5169ce3318ee admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:39.979414 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0d088d79-127f-4474-9ec6-5169ce3318ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:40.042265 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d088d79-127f-4474-9ec6-5169ce3318ee admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:40.045961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803077e4-439b-4c1b-81aa-55f246818b81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:40.046571 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803077e4-439b-4c1b-81aa-55f246818b81 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:40.046849 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803077e4-439b-4c1b-81aa-55f246818b81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:40.047121 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803077e4-439b-4c1b-81aa-55f246818b81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:40.056176 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803077e4-439b-4c1b-81aa-55f246818b81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:40.057636 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803077e4-439b-4c1b-81aa-55f246818b81 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:40.067507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1273/2573] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:09:39 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:09:40.143076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f79ee4-9355-43fb-b824-f0ec70e8dc98 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:40.153265 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f79ee4-9355-43fb-b824-f0ec70e8dc98 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:40.157301 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8664bb3-0b1f-4d1f-ae30-5357174ae6ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:40.157924 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8664bb3-0b1f-4d1f-ae30-5357174ae6ed service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:40.158204 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8664bb3-0b1f-4d1f-ae30-5357174ae6ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:40.158481 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8664bb3-0b1f-4d1f-ae30-5357174ae6ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:40.214478 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8664bb3-0b1f-4d1f-ae30-5357174ae6ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:40.215898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8664bb3-0b1f-4d1f-ae30-5357174ae6ed service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:40.224934 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1301/2574] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:09:40 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:09:40.923172 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803077e4-439b-4c1b-81aa-55f246818b81 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:40.924611 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-803077e4-439b-4c1b-81aa-55f246818b81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:40.987463 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803077e4-439b-4c1b-81aa-55f246818b81 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:40.991956 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c45e5-1c12-4891-a58f-d689fd3ce37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:40.993014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c45e5-1c12-4891-a58f-d689fd3ce37d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:40.993295 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c45e5-1c12-4891-a58f-d689fd3ce37d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:40.993565 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c45e5-1c12-4891-a58f-d689fd3ce37d admin admin] PATH_INFO: `/v3/users/1caaca5f82c045898c31c00449768c40` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:40.994722 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c45e5-1c12-4891-a58f-d689fd3ce37d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1caaca5f82c045898c31c00449768c40)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:40.996664 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c45e5-1c12-4891-a58f-d689fd3ce37d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:41.043182 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-da1c45e5-1c12-4891-a58f-d689fd3ce37d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1caaca5f82c045898c31c00449768c40'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:41.050461 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-da1c45e5-1c12-4891-a58f-d689fd3ce37d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1caaca5f82c045898c31c00449768c40'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:41.058035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:41.063562 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-da1c45e5-1c12-4891-a58f-d689fd3ce37d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1caaca5f82c045898c31c00449768c40'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:41.064360 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-da1c45e5-1c12-4891-a58f-d689fd3ce37d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1caaca5f82c045898c31c00449768c40'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:41.065503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-da1c45e5-1c12-4891-a58f-d689fd3ce37d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1caaca5f82c045898c31c00449768c40'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:41.077745 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1274/2575] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:09:40 2022] DELETE /identity/v3/users/1caaca5f82c045898c31c00449768c40 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:41.082234 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750fd03-f082-4a78-bc64-94e8cf1fa960 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:41.082713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750fd03-f082-4a78-bc64-94e8cf1fa960 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:41.082810 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750fd03-f082-4a78-bc64-94e8cf1fa960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:41.095750 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:41.102515 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5750fd03-f082-4a78-bc64-94e8cf1fa960 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:41.103702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5750fd03-f082-4a78-bc64-94e8cf1fa960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:41.161453 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1302/2576] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:41.171108 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c45e5-1c12-4891-a58f-d689fd3ce37d admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:41.173026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-da1c45e5-1c12-4891-a58f-d689fd3ce37d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:41.228406 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c45e5-1c12-4891-a58f-d689fd3ce37d admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:41.232007 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8968859-0f64-4ba6-856f-2451de239691 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:41.232641 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8968859-0f64-4ba6-856f-2451de239691 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:41.232923 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8968859-0f64-4ba6-856f-2451de239691 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:41.233194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8968859-0f64-4ba6-856f-2451de239691 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:41.242385 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8968859-0f64-4ba6-856f-2451de239691 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:41.243807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8968859-0f64-4ba6-856f-2451de239691 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:41.253587 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1275/2577] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:09:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:09:41.515417 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5750fd03-f082-4a78-bc64-94e8cf1fa960 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:41.516842 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5750fd03-f082-4a78-bc64-94e8cf1fa960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:41.528994 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5750fd03-f082-4a78-bc64-94e8cf1fa960 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:41.543996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be886db-9ff4-4287-82cc-3de7f16c261c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:41.544775 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be886db-9ff4-4287-82cc-3de7f16c261c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:41.544876 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be886db-9ff4-4287-82cc-3de7f16c261c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:41.545147 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be886db-9ff4-4287-82cc-3de7f16c261c admin admin] PATH_INFO: `/v3/projects/f11c9e2bbbd9430e82b54354e114ac94` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:41.546744 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be886db-9ff4-4287-82cc-3de7f16c261c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f11c9e2bbbd9430e82b54354e114ac94)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:41.549286 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be886db-9ff4-4287-82cc-3de7f16c261c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:41.597129 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5be886db-9ff4-4287-82cc-3de7f16c261c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f11c9e2bbbd9430e82b54354e114ac94', 'request_id': 'req-5be886db-9ff4-4287-82cc-3de7f16c261c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:41.604198 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5be886db-9ff4-4287-82cc-3de7f16c261c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f11c9e2bbbd9430e82b54354e114ac94', 'request_id': 'req-5be886db-9ff4-4287-82cc-3de7f16c261c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:41.612689 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5be886db-9ff4-4287-82cc-3de7f16c261c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f11c9e2bbbd9430e82b54354e114ac94', 'request_id': 'req-5be886db-9ff4-4287-82cc-3de7f16c261c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:41.623991 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5be886db-9ff4-4287-82cc-3de7f16c261c admin admin] The token cache is being invalidate because project f11c9e2bbbd9430e82b54354e114ac94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:09:41.624280 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5be886db-9ff4-4287-82cc-3de7f16c261c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:41.624945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5be886db-9ff4-4287-82cc-3de7f16c261c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:41.626457 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1303/2578] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:09:41 2022] DELETE /identity/v3/projects/f11c9e2bbbd9430e82b54354e114ac94 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:41.630494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8968859-0f64-4ba6-856f-2451de239691 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:41.639976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8968859-0f64-4ba6-856f-2451de239691 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:41.698839 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc471159-8524-48da-b702-de712beb76e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:41.699558 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc471159-8524-48da-b702-de712beb76e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:41.699835 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc471159-8524-48da-b702-de712beb76e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:41.700108 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc471159-8524-48da-b702-de712beb76e9 admin admin] PATH_INFO: `/v3/users/f19a1a27f9b84689b2bacf1902b7fa42` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:41.701322 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc471159-8524-48da-b702-de712beb76e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f19a1a27f9b84689b2bacf1902b7fa42)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:41.703478 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc471159-8524-48da-b702-de712beb76e9 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:41.752155 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-cc471159-8524-48da-b702-de712beb76e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f19a1a27f9b84689b2bacf1902b7fa42'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:41.759725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-cc471159-8524-48da-b702-de712beb76e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f19a1a27f9b84689b2bacf1902b7fa42'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:41.772155 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-cc471159-8524-48da-b702-de712beb76e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f19a1a27f9b84689b2bacf1902b7fa42'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:41.772891 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-cc471159-8524-48da-b702-de712beb76e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f19a1a27f9b84689b2bacf1902b7fa42'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:41.773561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-cc471159-8524-48da-b702-de712beb76e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f19a1a27f9b84689b2bacf1902b7fa42'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:41.787262 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1276/2579] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:09:41 2022] DELETE /identity/v3/users/f19a1a27f9b84689b2bacf1902b7fa42 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:42.055974 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be886db-9ff4-4287-82cc-3de7f16c261c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:42.057438 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5be886db-9ff4-4287-82cc-3de7f16c261c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:42.072223 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be886db-9ff4-4287-82cc-3de7f16c261c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:42.128601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5265856-1efe-42e8-838e-480d044876db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:42.129204 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5265856-1efe-42e8-838e-480d044876db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:42.129430 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5265856-1efe-42e8-838e-480d044876db admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:42.129702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5265856-1efe-42e8-838e-480d044876db admin admin] PATH_INFO: `/v3/projects/085487091d314139bb907fb39acb1900` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:42.130835 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5265856-1efe-42e8-838e-480d044876db admin admin] RBAC: Authorizing `identity:delete_project(project_id=085487091d314139bb907fb39acb1900)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:42.132778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5265856-1efe-42e8-838e-480d044876db admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:42.165900 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:42.179145 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d5265856-1efe-42e8-838e-480d044876db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '085487091d314139bb907fb39acb1900', 'request_id': 'req-d5265856-1efe-42e8-838e-480d044876db'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:42.184693 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d5265856-1efe-42e8-838e-480d044876db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '085487091d314139bb907fb39acb1900', 'request_id': 'req-d5265856-1efe-42e8-838e-480d044876db'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:42.192134 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d5265856-1efe-42e8-838e-480d044876db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '085487091d314139bb907fb39acb1900', 'request_id': 'req-d5265856-1efe-42e8-838e-480d044876db'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:42.203303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d5265856-1efe-42e8-838e-480d044876db admin admin] The token cache is being invalidate because project 085487091d314139bb907fb39acb1900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:09:42.204200 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d5265856-1efe-42e8-838e-480d044876db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:42.204838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d5265856-1efe-42e8-838e-480d044876db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:42.206335 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1304/2580] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:09:42 2022] DELETE /identity/v3/projects/085487091d314139bb907fb39acb1900 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:42.211196 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6114ea-1f79-47b8-845c-2945cecd1c1e None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:42.211703 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6114ea-1f79-47b8-845c-2945cecd1c1e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:42.211836 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6114ea-1f79-47b8-845c-2945cecd1c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:42.225621 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:42.232413 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-cc6114ea-1f79-47b8-845c-2945cecd1c1e None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:42.234289 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-cc6114ea-1f79-47b8-845c-2945cecd1c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:42.292620 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1277/2581] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:42.299901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d064f58-c0f3-4fe6-ac11-9ddfb3cd7f78 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:42.300185 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d064f58-c0f3-4fe6-ac11-9ddfb3cd7f78 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:42.300491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d064f58-c0f3-4fe6-ac11-9ddfb3cd7f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:42.320968 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1d064f58-c0f3-4fe6-ac11-9ddfb3cd7f78 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:42.322149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1d064f58-c0f3-4fe6-ac11-9ddfb3cd7f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:42.330605 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1305/2582] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:42.334598 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6114ea-1f79-47b8-845c-2945cecd1c1e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:42.343949 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6114ea-1f79-47b8-845c-2945cecd1c1e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:42.348544 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34eaeea-6e5e-4254-a41c-59aa3bcf718c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:42.349392 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34eaeea-6e5e-4254-a41c-59aa3bcf718c admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:42.349666 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34eaeea-6e5e-4254-a41c-59aa3bcf718c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:42.349931 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34eaeea-6e5e-4254-a41c-59aa3bcf718c admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:42.350616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34eaeea-6e5e-4254-a41c-59aa3bcf718c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:09:42.350913 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34eaeea-6e5e-4254-a41c-59aa3bcf718c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:42.352512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34eaeea-6e5e-4254-a41c-59aa3bcf718c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:42.364052 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1278/2583] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 13:09:42.368140 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d064f58-c0f3-4fe6-ac11-9ddfb3cd7f78 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:42.377748 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d064f58-c0f3-4fe6-ac11-9ddfb3cd7f78 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:42.382402 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa26b2-38a4-451c-80cf-94b4f6f50a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:42.382949 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa26b2-38a4-451c-80cf-94b4f6f50a23 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:42.383227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa26b2-38a4-451c-80cf-94b4f6f50a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:42.383498 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa26b2-38a4-451c-80cf-94b4f6f50a23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:42.384190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa26b2-38a4-451c-80cf-94b4f6f50a23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:42.386108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa26b2-38a4-451c-80cf-94b4f6f50a23 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:42.418564 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1306/2584] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:09:42 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:09:42.420089 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34eaeea-6e5e-4254-a41c-59aa3bcf718c admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:42.430609 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34eaeea-6e5e-4254-a41c-59aa3bcf718c admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:42.484656 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b5f72-7cc1-4bc1-ab72-384878bc4fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:42.485743 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b5f72-7cc1-4bc1-ab72-384878bc4fbe admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:42.486018 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b5f72-7cc1-4bc1-ab72-384878bc4fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:42.486285 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b5f72-7cc1-4bc1-ab72-384878bc4fbe admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:42.486938 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b5f72-7cc1-4bc1-ab72-384878bc4fbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:42.488853 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b5f72-7cc1-4bc1-ab72-384878bc4fbe admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:42.509320 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1279/2585] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:09:42 2022] POST /identity/v3/users => generated 283 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:09:42.513496 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa26b2-38a4-451c-80cf-94b4f6f50a23 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:42.523640 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa26b2-38a4-451c-80cf-94b4f6f50a23 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:42.528255 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1f7ee-84da-4e52-b37c-fe82318b265f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:42.528885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1f7ee-84da-4e52-b37c-fe82318b265f admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:42.529165 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1f7ee-84da-4e52-b37c-fe82318b265f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:42.529438 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1f7ee-84da-4e52-b37c-fe82318b265f admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:42.530124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1f7ee-84da-4e52-b37c-fe82318b265f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:42.531728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1f7ee-84da-4e52-b37c-fe82318b265f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:42.541952 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1307/2586] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:09:42.545750 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b5f72-7cc1-4bc1-ab72-384878bc4fbe admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:42.557621 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b5f72-7cc1-4bc1-ab72-384878bc4fbe admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:42.562173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2914cd-5ae3-46f3-9c17-5b05726ea730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:42.562770 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2914cd-5ae3-46f3-9c17-5b05726ea730 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:42.563626 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2914cd-5ae3-46f3-9c17-5b05726ea730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:42.563896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2914cd-5ae3-46f3-9c17-5b05726ea730 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:42.564630 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2914cd-5ae3-46f3-9c17-5b05726ea730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:42.566245 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2914cd-5ae3-46f3-9c17-5b05726ea730 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:42.576666 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1280/2587] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:09:42.580733 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1f7ee-84da-4e52-b37c-fe82318b265f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:42.590915 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1f7ee-84da-4e52-b37c-fe82318b265f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:42.595507 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29298d14-83ea-473a-bac1-639b12a2b728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:42.596161 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29298d14-83ea-473a-bac1-639b12a2b728 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:42.596479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29298d14-83ea-473a-bac1-639b12a2b728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:42.596751 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29298d14-83ea-473a-bac1-639b12a2b728 admin admin] PATH_INFO: `/v3/projects/6bd7234940c949038d532a9a0ac3c2bc/users/5953fa87df04465d9867fa284084ccfe/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:42.612500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29298d14-83ea-473a-bac1-639b12a2b728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bd7234940c949038d532a9a0ac3c2bc, user_id=5953fa87df04465d9867fa284084ccfe, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:42.616804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29298d14-83ea-473a-bac1-639b12a2b728 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:42.634242 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1308/2588] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:09:42 2022] PUT /identity/v3/projects/6bd7234940c949038d532a9a0ac3c2bc/users/5953fa87df04465d9867fa284084ccfe/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:42.643933 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adf857-59e0-4e54-a28c-e4899ea2f328 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:42.644216 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adf857-59e0-4e54-a28c-e4899ea2f328 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:42.644518 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adf857-59e0-4e54-a28c-e4899ea2f328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:42.670766 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c6adf857-59e0-4e54-a28c-e4899ea2f328 None None] MFA Rules not processed for user `5953fa87df04465d9867fa284084ccfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:42.671943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c6adf857-59e0-4e54-a28c-e4899ea2f328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:42.736468 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1281/2589] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:42.740920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29298d14-83ea-473a-bac1-639b12a2b728 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:42.801175 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29298d14-83ea-473a-bac1-639b12a2b728 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:42.805726 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5b3f1-bdc5-4f19-8973-0106a7625ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:42.806328 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5b3f1-bdc5-4f19-8973-0106a7625ce1 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:42.806607 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5b3f1-bdc5-4f19-8973-0106a7625ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:42.806879 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5b3f1-bdc5-4f19-8973-0106a7625ce1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:42.807527 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5b3f1-bdc5-4f19-8973-0106a7625ce1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:42.809446 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5b3f1-bdc5-4f19-8973-0106a7625ce1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:42.835124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1309/2590] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:09:42 2022] POST /identity/v3/projects => generated 363 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:09:42.839040 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6adf857-59e0-4e54-a28c-e4899ea2f328 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:42.844842 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:42.851975 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6adf857-59e0-4e54-a28c-e4899ea2f328 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:42.915540 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe74cc4-21bd-4a33-90ba-49a29cfce667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:42.916159 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe74cc4-21bd-4a33-90ba-49a29cfce667 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:42.916872 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe74cc4-21bd-4a33-90ba-49a29cfce667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:42.916872 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe74cc4-21bd-4a33-90ba-49a29cfce667 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:42.917402 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe74cc4-21bd-4a33-90ba-49a29cfce667 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:42.920518 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe74cc4-21bd-4a33-90ba-49a29cfce667 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:42.943901 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1282/2591] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:09:42 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:09:42.947723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5b3f1-bdc5-4f19-8973-0106a7625ce1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:42.958482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5b3f1-bdc5-4f19-8973-0106a7625ce1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:42.963156 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef79c2-9e6b-4288-a215-bcc7d7081354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:42.963767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef79c2-9e6b-4288-a215-bcc7d7081354 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:42.964051 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef79c2-9e6b-4288-a215-bcc7d7081354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:42.964390 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef79c2-9e6b-4288-a215-bcc7d7081354 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:42.965088 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef79c2-9e6b-4288-a215-bcc7d7081354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:42.966664 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef79c2-9e6b-4288-a215-bcc7d7081354 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:42.981379 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1310/2592] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:09:42.982312 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe74cc4-21bd-4a33-90ba-49a29cfce667 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:42.992969 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe74cc4-21bd-4a33-90ba-49a29cfce667 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:42.998231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c5729-3060-4423-b1f0-c5f76ba8afc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:42.998900 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c5729-3060-4423-b1f0-c5f76ba8afc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:42.999206 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c5729-3060-4423-b1f0-c5f76ba8afc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:42.999461 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c5729-3060-4423-b1f0-c5f76ba8afc5 admin admin] PATH_INFO: `/v3/projects/a45de26351054ec28ba0e43ee9a70a3d/users/08a7884cded14138b23e9142cc8e1021/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:43.017508 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c5729-3060-4423-b1f0-c5f76ba8afc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a45de26351054ec28ba0e43ee9a70a3d, user_id=08a7884cded14138b23e9142cc8e1021, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:43.020189 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c5729-3060-4423-b1f0-c5f76ba8afc5 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:43.033751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1283/2593] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:09:42 2022] PUT /identity/v3/projects/a45de26351054ec28ba0e43ee9a70a3d/users/08a7884cded14138b23e9142cc8e1021/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:09:43.038062 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef79c2-9e6b-4288-a215-bcc7d7081354 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:43.048705 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef79c2-9e6b-4288-a215-bcc7d7081354 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:43.079843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:43.114068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0e549-b942-4d9e-9159-cdc4d860b393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:43.114736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0e549-b942-4d9e-9159-cdc4d860b393 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:43.115018 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0e549-b942-4d9e-9159-cdc4d860b393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:43.115320 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0e549-b942-4d9e-9159-cdc4d860b393 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:43.116051 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a0e549-b942-4d9e-9159-cdc4d860b393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:43.117677 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a0e549-b942-4d9e-9159-cdc4d860b393 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:43.127939 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1311/2594] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:09:43.132914 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c5729-3060-4423-b1f0-c5f76ba8afc5 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:43.148959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c5729-3060-4423-b1f0-c5f76ba8afc5 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:43.153886 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa6a68-ee00-4583-a638-930d177d15ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:43.154625 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa6a68-ee00-4583-a638-930d177d15ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:43.154978 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa6a68-ee00-4583-a638-930d177d15ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:43.155324 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa6a68-ee00-4583-a638-930d177d15ed admin admin] PATH_INFO: `/v3/projects/a45de26351054ec28ba0e43ee9a70a3d/users/08a7884cded14138b23e9142cc8e1021/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:43.157869 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa6a68-ee00-4583-a638-930d177d15ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=a45de26351054ec28ba0e43ee9a70a3d, user_id=08a7884cded14138b23e9142cc8e1021, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:43.160568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa6a68-ee00-4583-a638-930d177d15ed admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:43.170798 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1284/2595] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:09:43 2022] PUT /identity/v3/projects/a45de26351054ec28ba0e43ee9a70a3d/users/08a7884cded14138b23e9142cc8e1021/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:43.182167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559afa52-a795-4e19-92b9-5cbfd36a39ae None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:43.182612 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559afa52-a795-4e19-92b9-5cbfd36a39ae None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:43.182726 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559afa52-a795-4e19-92b9-5cbfd36a39ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:43.210745 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-559afa52-a795-4e19-92b9-5cbfd36a39ae None None] MFA Rules not processed for user `08a7884cded14138b23e9142cc8e1021`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:43.212095 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-559afa52-a795-4e19-92b9-5cbfd36a39ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:43.279195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1312/2596] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:43.291135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa6a68-ee00-4583-a638-930d177d15ed admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:43.292966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-63fa6a68-ee00-4583-a638-930d177d15ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:43.350719 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa6a68-ee00-4583-a638-930d177d15ed admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:43.354943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc92d98-feb9-4e55-93b2-41589ac8b441 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:43.355556 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc92d98-feb9-4e55-93b2-41589ac8b441 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:43.355901 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc92d98-feb9-4e55-93b2-41589ac8b441 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:43.356173 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc92d98-feb9-4e55-93b2-41589ac8b441 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:43.366084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc92d98-feb9-4e55-93b2-41589ac8b441 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:43.367810 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc92d98-feb9-4e55-93b2-41589ac8b441 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:43.378214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1285/2597] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:09:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:09:43.454551 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559afa52-a795-4e19-92b9-5cbfd36a39ae None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:43.464548 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559afa52-a795-4e19-92b9-5cbfd36a39ae None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:43.468443 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92585b5d-b462-497c-b3d7-725e28da31dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:43.469151 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92585b5d-b462-497c-b3d7-725e28da31dd service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:43.469716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92585b5d-b462-497c-b3d7-725e28da31dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:43.470000 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92585b5d-b462-497c-b3d7-725e28da31dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:43.531500 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92585b5d-b462-497c-b3d7-725e28da31dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:43.532997 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92585b5d-b462-497c-b3d7-725e28da31dd service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:43.541883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1313/2598] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:09:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:09:44.313388 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc92d98-feb9-4e55-93b2-41589ac8b441 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:44.314958 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7bc92d98-feb9-4e55-93b2-41589ac8b441 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:44.373814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:44.378801 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc92d98-feb9-4e55-93b2-41589ac8b441 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:44.384462 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb142bf-41e6-4e29-8f0c-24bad12d43cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:44.385091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb142bf-41e6-4e29-8f0c-24bad12d43cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:44.385370 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb142bf-41e6-4e29-8f0c-24bad12d43cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:44.385642 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb142bf-41e6-4e29-8f0c-24bad12d43cd admin admin] PATH_INFO: `/v3/users/ab7292a8d3554a64818e5cfc4d8efbc0` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:44.386811 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb142bf-41e6-4e29-8f0c-24bad12d43cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab7292a8d3554a64818e5cfc4d8efbc0)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:44.388808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb142bf-41e6-4e29-8f0c-24bad12d43cd admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:44.433156 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6fb142bf-41e6-4e29-8f0c-24bad12d43cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab7292a8d3554a64818e5cfc4d8efbc0'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:44.440432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6fb142bf-41e6-4e29-8f0c-24bad12d43cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab7292a8d3554a64818e5cfc4d8efbc0'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:44.451982 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6fb142bf-41e6-4e29-8f0c-24bad12d43cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab7292a8d3554a64818e5cfc4d8efbc0'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:44.452661 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6fb142bf-41e6-4e29-8f0c-24bad12d43cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab7292a8d3554a64818e5cfc4d8efbc0'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:44.453272 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-6fb142bf-41e6-4e29-8f0c-24bad12d43cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab7292a8d3554a64818e5cfc4d8efbc0'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:44.466742 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1286/2599] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:09:44 2022] DELETE /identity/v3/users/ab7292a8d3554a64818e5cfc4d8efbc0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:44.471239 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27941cf-630e-4174-825e-75f9ea29554f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:44.471742 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27941cf-630e-4174-825e-75f9ea29554f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:44.471836 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27941cf-630e-4174-825e-75f9ea29554f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:44.490468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f27941cf-630e-4174-825e-75f9ea29554f None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:44.491652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f27941cf-630e-4174-825e-75f9ea29554f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:44.537769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:44.552244 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1314/2600] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:44.562056 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb142bf-41e6-4e29-8f0c-24bad12d43cd admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:44.564044 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6fb142bf-41e6-4e29-8f0c-24bad12d43cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:44.624992 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb142bf-41e6-4e29-8f0c-24bad12d43cd admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:44.629549 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363dd7b8-7778-4075-80b7-ed92a394c6a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:44.630286 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363dd7b8-7778-4075-80b7-ed92a394c6a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:44.630438 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363dd7b8-7778-4075-80b7-ed92a394c6a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:44.630708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363dd7b8-7778-4075-80b7-ed92a394c6a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:44.639881 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363dd7b8-7778-4075-80b7-ed92a394c6a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:44.641986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363dd7b8-7778-4075-80b7-ed92a394c6a7 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:44.651358 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1287/2601] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:09:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:09:45.040696 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27941cf-630e-4174-825e-75f9ea29554f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:45.042143 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f27941cf-630e-4174-825e-75f9ea29554f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:45.067551 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27941cf-630e-4174-825e-75f9ea29554f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:45.073814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695da51-8253-4675-b509-c80b1d77ed1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:45.074573 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695da51-8253-4675-b509-c80b1d77ed1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:45.074929 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695da51-8253-4675-b509-c80b1d77ed1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:45.075296 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695da51-8253-4675-b509-c80b1d77ed1c admin admin] PATH_INFO: `/v3/projects/7221efb3614a41748e3cadfc7a9326fd` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:45.076758 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695da51-8253-4675-b509-c80b1d77ed1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7221efb3614a41748e3cadfc7a9326fd)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:45.079063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695da51-8253-4675-b509-c80b1d77ed1c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:45.160039 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b695da51-8253-4675-b509-c80b1d77ed1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7221efb3614a41748e3cadfc7a9326fd', 'request_id': 'req-b695da51-8253-4675-b509-c80b1d77ed1c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:45.167175 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b695da51-8253-4675-b509-c80b1d77ed1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7221efb3614a41748e3cadfc7a9326fd', 'request_id': 'req-b695da51-8253-4675-b509-c80b1d77ed1c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:45.175962 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b695da51-8253-4675-b509-c80b1d77ed1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7221efb3614a41748e3cadfc7a9326fd', 'request_id': 'req-b695da51-8253-4675-b509-c80b1d77ed1c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:45.190270 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b695da51-8253-4675-b509-c80b1d77ed1c admin admin] The token cache is being invalidate because project 7221efb3614a41748e3cadfc7a9326fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:09:45.190563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b695da51-8253-4675-b509-c80b1d77ed1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:45.191248 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b695da51-8253-4675-b509-c80b1d77ed1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:45.192830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1315/2602] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:09:45 2022] DELETE /identity/v3/projects/7221efb3614a41748e3cadfc7a9326fd => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:45.196957 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363dd7b8-7778-4075-80b7-ed92a394c6a7 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:45.206678 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363dd7b8-7778-4075-80b7-ed92a394c6a7 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:45.264433 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc73fee0-ba92-4c46-8dc0-ef41b91d4505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:45.265097 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73fee0-ba92-4c46-8dc0-ef41b91d4505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:45.265376 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73fee0-ba92-4c46-8dc0-ef41b91d4505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:45.265645 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73fee0-ba92-4c46-8dc0-ef41b91d4505 admin admin] PATH_INFO: `/v3/users/a717e20ac4714379ba2f8463d32150f5` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:45.266823 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc73fee0-ba92-4c46-8dc0-ef41b91d4505 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a717e20ac4714379ba2f8463d32150f5)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:45.268876 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc73fee0-ba92-4c46-8dc0-ef41b91d4505 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:45.414715 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:45.441814 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fc73fee0-ba92-4c46-8dc0-ef41b91d4505 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a717e20ac4714379ba2f8463d32150f5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:45.448443 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fc73fee0-ba92-4c46-8dc0-ef41b91d4505 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a717e20ac4714379ba2f8463d32150f5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:45.460364 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fc73fee0-ba92-4c46-8dc0-ef41b91d4505 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a717e20ac4714379ba2f8463d32150f5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:45.461021 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fc73fee0-ba92-4c46-8dc0-ef41b91d4505 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a717e20ac4714379ba2f8463d32150f5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:45.461614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-fc73fee0-ba92-4c46-8dc0-ef41b91d4505 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a717e20ac4714379ba2f8463d32150f5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:45.475038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1288/2603] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:09:45 2022] DELETE /identity/v3/users/a717e20ac4714379ba2f8463d32150f5 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:45.748211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695da51-8253-4675-b509-c80b1d77ed1c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:45.749744 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b695da51-8253-4675-b509-c80b1d77ed1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:45.764355 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695da51-8253-4675-b509-c80b1d77ed1c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:45.830669 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5fe26-84f9-4b32-b239-0f381f9b2921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:45.831318 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5fe26-84f9-4b32-b239-0f381f9b2921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:45.831603 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5fe26-84f9-4b32-b239-0f381f9b2921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:45.831877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5fe26-84f9-4b32-b239-0f381f9b2921 admin admin] PATH_INFO: `/v3/projects/eee8a7fcac9b4789b005753fe674afa1` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:45.833058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b5fe26-84f9-4b32-b239-0f381f9b2921 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eee8a7fcac9b4789b005753fe674afa1)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:45.834967 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b5fe26-84f9-4b32-b239-0f381f9b2921 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:45.879589 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-52b5fe26-84f9-4b32-b239-0f381f9b2921 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eee8a7fcac9b4789b005753fe674afa1', 'request_id': 'req-52b5fe26-84f9-4b32-b239-0f381f9b2921'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:45.884936 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-52b5fe26-84f9-4b32-b239-0f381f9b2921 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eee8a7fcac9b4789b005753fe674afa1', 'request_id': 'req-52b5fe26-84f9-4b32-b239-0f381f9b2921'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:45.892525 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-52b5fe26-84f9-4b32-b239-0f381f9b2921 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eee8a7fcac9b4789b005753fe674afa1', 'request_id': 'req-52b5fe26-84f9-4b32-b239-0f381f9b2921'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:45.903522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-52b5fe26-84f9-4b32-b239-0f381f9b2921 admin admin] The token cache is being invalidate because project eee8a7fcac9b4789b005753fe674afa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:09:45.903809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-52b5fe26-84f9-4b32-b239-0f381f9b2921 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:45.904525 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-52b5fe26-84f9-4b32-b239-0f381f9b2921 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:45.906068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1316/2604] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:09:45 2022] DELETE /identity/v3/projects/eee8a7fcac9b4789b005753fe674afa1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:45.911100 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c54501-1f52-4a0f-b639-f89efaccd44e None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:45.911446 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c54501-1f52-4a0f-b639-f89efaccd44e None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:45.911918 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c54501-1f52-4a0f-b639-f89efaccd44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:45.931886 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-79c54501-1f52-4a0f-b639-f89efaccd44e None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:45.933153 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-79c54501-1f52-4a0f-b639-f89efaccd44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:45.991147 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1289/2605] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:45.998704 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303c8ec-9fb9-45f9-b381-60db1aef3d55 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:45.998991 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303c8ec-9fb9-45f9-b381-60db1aef3d55 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:45.999267 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303c8ec-9fb9-45f9-b381-60db1aef3d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:46.008515 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:46.021746 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6303c8ec-9fb9-45f9-b381-60db1aef3d55 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:46.023791 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6303c8ec-9fb9-45f9-b381-60db1aef3d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:46.031799 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1317/2606] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:46.035979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c54501-1f52-4a0f-b639-f89efaccd44e None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:46.045450 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c54501-1f52-4a0f-b639-f89efaccd44e None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:46.050030 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f51e4c-9ae8-4921-bcd6-45b237f0eb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:46.050632 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f51e4c-9ae8-4921-bcd6-45b237f0eb84 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:46.050909 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f51e4c-9ae8-4921-bcd6-45b237f0eb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:46.051178 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f51e4c-9ae8-4921-bcd6-45b237f0eb84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:46.051860 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f51e4c-9ae8-4921-bcd6-45b237f0eb84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:09:46.052156 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f51e4c-9ae8-4921-bcd6-45b237f0eb84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:46.053777 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f51e4c-9ae8-4921-bcd6-45b237f0eb84 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:46.065540 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1290/2607] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 13:09:46.069578 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303c8ec-9fb9-45f9-b381-60db1aef3d55 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:46.079635 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303c8ec-9fb9-45f9-b381-60db1aef3d55 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:46.084588 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820f467-4051-4648-8697-7cecd30506d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:46.085277 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820f467-4051-4648-8697-7cecd30506d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:46.085565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820f467-4051-4648-8697-7cecd30506d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:46.085836 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820f467-4051-4648-8697-7cecd30506d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:46.086505 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820f467-4051-4648-8697-7cecd30506d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:46.088419 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820f467-4051-4648-8697-7cecd30506d7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:46.115442 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1318/2608] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:09:46 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:09:46.119739 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f51e4c-9ae8-4921-bcd6-45b237f0eb84 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:46.130005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f51e4c-9ae8-4921-bcd6-45b237f0eb84 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:46.185526 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3af1de-e583-408e-9e7a-5a608bb2e531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:46.186141 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3af1de-e583-408e-9e7a-5a608bb2e531 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:46.186421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3af1de-e583-408e-9e7a-5a608bb2e531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:46.186692 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3af1de-e583-408e-9e7a-5a608bb2e531 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:46.187361 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3af1de-e583-408e-9e7a-5a608bb2e531 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:46.189830 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3af1de-e583-408e-9e7a-5a608bb2e531 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:46.211022 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1291/2609] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:09:46 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:09:46.215237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820f467-4051-4648-8697-7cecd30506d7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:46.225957 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820f467-4051-4648-8697-7cecd30506d7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:46.231244 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c2ce7c-7283-4cc4-8e2f-77414c4ab2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:46.231885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2ce7c-7283-4cc4-8e2f-77414c4ab2d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:46.232172 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2ce7c-7283-4cc4-8e2f-77414c4ab2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:46.232486 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2ce7c-7283-4cc4-8e2f-77414c4ab2d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:46.233236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c2ce7c-7283-4cc4-8e2f-77414c4ab2d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:46.235386 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c2ce7c-7283-4cc4-8e2f-77414c4ab2d5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:46.245771 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1319/2610] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:09:46.249592 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3af1de-e583-408e-9e7a-5a608bb2e531 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:46.259991 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3af1de-e583-408e-9e7a-5a608bb2e531 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:46.264834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173c86e-f070-4bd4-acc6-8f01db0ae50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:46.265507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173c86e-f070-4bd4-acc6-8f01db0ae50a admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:46.265851 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173c86e-f070-4bd4-acc6-8f01db0ae50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:46.266187 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173c86e-f070-4bd4-acc6-8f01db0ae50a admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:46.266943 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7173c86e-f070-4bd4-acc6-8f01db0ae50a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:46.268589 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7173c86e-f070-4bd4-acc6-8f01db0ae50a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:46.279061 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1292/2611] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:09:46.283140 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c2ce7c-7283-4cc4-8e2f-77414c4ab2d5 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:46.293042 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c2ce7c-7283-4cc4-8e2f-77414c4ab2d5 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:46.298499 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2668537d-d40f-437b-aafb-08146c1db512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:46.299155 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668537d-d40f-437b-aafb-08146c1db512 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:46.299434 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668537d-d40f-437b-aafb-08146c1db512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:46.299709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668537d-d40f-437b-aafb-08146c1db512 admin admin] PATH_INFO: `/v3/projects/7482fc21505f4ba7970034e9e6d807b8/users/4d2eaaca53414894a263418b6917a8ce/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:46.314934 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2668537d-d40f-437b-aafb-08146c1db512 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7482fc21505f4ba7970034e9e6d807b8, user_id=4d2eaaca53414894a263418b6917a8ce, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:46.317664 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2668537d-d40f-437b-aafb-08146c1db512 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:46.330675 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1320/2612] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:09:46 2022] PUT /identity/v3/projects/7482fc21505f4ba7970034e9e6d807b8/users/4d2eaaca53414894a263418b6917a8ce/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:09:46.341786 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95c9a6-60db-487e-ae38-2b2577059ed1 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:46.342067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95c9a6-60db-487e-ae38-2b2577059ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:46.342336 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95c9a6-60db-487e-ae38-2b2577059ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:46.350811 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:46.371647 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-be95c9a6-60db-487e-ae38-2b2577059ed1 None None] MFA Rules not processed for user `4d2eaaca53414894a263418b6917a8ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:46.372883 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-be95c9a6-60db-487e-ae38-2b2577059ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:46.437342 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1293/2613] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:46.442807 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2668537d-d40f-437b-aafb-08146c1db512 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:46.505851 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2668537d-d40f-437b-aafb-08146c1db512 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:46.511067 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce037261-09d8-436b-8f8f-3ad6da774e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:46.511677 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce037261-09d8-436b-8f8f-3ad6da774e45 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:46.511960 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce037261-09d8-436b-8f8f-3ad6da774e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:46.512236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce037261-09d8-436b-8f8f-3ad6da774e45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:46.513055 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce037261-09d8-436b-8f8f-3ad6da774e45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:46.515059 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce037261-09d8-436b-8f8f-3ad6da774e45 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:46.542331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1321/2614] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:09:46 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:09:46.546657 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95c9a6-60db-487e-ae38-2b2577059ed1 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:46.558151 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95c9a6-60db-487e-ae38-2b2577059ed1 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:46.626661 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c4f77-bee6-4c75-8f65-929406c2bb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:46.628477 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c4f77-bee6-4c75-8f65-929406c2bb15 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:46.628951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c4f77-bee6-4c75-8f65-929406c2bb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:46.629537 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c4f77-bee6-4c75-8f65-929406c2bb15 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:46.630521 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6c4f77-bee6-4c75-8f65-929406c2bb15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:46.633691 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6c4f77-bee6-4c75-8f65-929406c2bb15 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:46.658661 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1294/2615] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:09:46 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:09:46.662610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce037261-09d8-436b-8f8f-3ad6da774e45 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:46.673561 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce037261-09d8-436b-8f8f-3ad6da774e45 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:46.678253 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914f962-cf1b-4e18-9fbd-f859471053ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:46.678890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914f962-cf1b-4e18-9fbd-f859471053ba admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:46.679166 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914f962-cf1b-4e18-9fbd-f859471053ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:46.679436 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914f962-cf1b-4e18-9fbd-f859471053ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:46.680124 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6914f962-cf1b-4e18-9fbd-f859471053ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:46.681734 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6914f962-cf1b-4e18-9fbd-f859471053ba admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:46.693114 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1322/2616] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:09:46.700501 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c4f77-bee6-4c75-8f65-929406c2bb15 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:46.710904 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c4f77-bee6-4c75-8f65-929406c2bb15 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:46.715858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267aaf5-e4b7-4dd5-9af3-462307c19741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:46.716515 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267aaf5-e4b7-4dd5-9af3-462307c19741 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:46.716838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267aaf5-e4b7-4dd5-9af3-462307c19741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:46.717066 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267aaf5-e4b7-4dd5-9af3-462307c19741 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:46.717747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5267aaf5-e4b7-4dd5-9af3-462307c19741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:46.719877 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5267aaf5-e4b7-4dd5-9af3-462307c19741 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:46.730786 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1295/2617] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:09:46.735603 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914f962-cf1b-4e18-9fbd-f859471053ba admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:46.745568 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914f962-cf1b-4e18-9fbd-f859471053ba admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:46.750858 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af2a16-9921-4099-9af5-ff471fd6ca10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:46.751439 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af2a16-9921-4099-9af5-ff471fd6ca10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:46.751724 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af2a16-9921-4099-9af5-ff471fd6ca10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:46.752003 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af2a16-9921-4099-9af5-ff471fd6ca10 admin admin] PATH_INFO: `/v3/projects/82ff19796ea641b191c4bfa58fa3484e/users/7d6cf24f00124aa6b23a2c8e63eadd5d/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:46.767007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af2a16-9921-4099-9af5-ff471fd6ca10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ff19796ea641b191c4bfa58fa3484e, user_id=7d6cf24f00124aa6b23a2c8e63eadd5d, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:46.769697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af2a16-9921-4099-9af5-ff471fd6ca10 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:46.787637 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1323/2618] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:09:46 2022] PUT /identity/v3/projects/82ff19796ea641b191c4bfa58fa3484e/users/7d6cf24f00124aa6b23a2c8e63eadd5d/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:46.798269 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437c5d5-3114-4718-85cf-5ca0daa25f15 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:46.798557 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437c5d5-3114-4718-85cf-5ca0daa25f15 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:46.798833 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437c5d5-3114-4718-85cf-5ca0daa25f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:46.829298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-1437c5d5-3114-4718-85cf-5ca0daa25f15 None None] MFA Rules not processed for user `7d6cf24f00124aa6b23a2c8e63eadd5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:46.830817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-1437c5d5-3114-4718-85cf-5ca0daa25f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:46.894061 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1296/2619] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:46.898654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af2a16-9921-4099-9af5-ff471fd6ca10 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:46.948820 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:46.961958 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af2a16-9921-4099-9af5-ff471fd6ca10 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:46.967095 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d59198-895e-4b1d-8e04-cb09ddfe110c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:46.967843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d59198-895e-4b1d-8e04-cb09ddfe110c admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:46.967986 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d59198-895e-4b1d-8e04-cb09ddfe110c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:46.968241 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d59198-895e-4b1d-8e04-cb09ddfe110c admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:46.969545 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d59198-895e-4b1d-8e04-cb09ddfe110c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:46.971437 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d59198-895e-4b1d-8e04-cb09ddfe110c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:46.997526 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1324/2620] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:09:46 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:09:47.001536 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437c5d5-3114-4718-85cf-5ca0daa25f15 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:47.007013 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:47.014247 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437c5d5-3114-4718-85cf-5ca0daa25f15 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:47.069289 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b3ac3-0284-42f3-b591-b9f475fffd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:47.069879 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b3ac3-0284-42f3-b591-b9f475fffd03 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:47.070161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b3ac3-0284-42f3-b591-b9f475fffd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:47.070434 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b3ac3-0284-42f3-b591-b9f475fffd03 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:47.071098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b3ac3-0284-42f3-b591-b9f475fffd03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:47.073056 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b3ac3-0284-42f3-b591-b9f475fffd03 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:47.094751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1297/2621] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:09:47 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:09:47.098633 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d59198-895e-4b1d-8e04-cb09ddfe110c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:47.109307 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d59198-895e-4b1d-8e04-cb09ddfe110c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:47.114013 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496ebc6-48a8-48b3-9036-da0e6b3130f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:47.114649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496ebc6-48a8-48b3-9036-da0e6b3130f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:47.114930 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496ebc6-48a8-48b3-9036-da0e6b3130f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:47.115206 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496ebc6-48a8-48b3-9036-da0e6b3130f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:47.115899 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496ebc6-48a8-48b3-9036-da0e6b3130f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:47.117585 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496ebc6-48a8-48b3-9036-da0e6b3130f2 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:47.128433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1325/2622] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:09:47.132470 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b3ac3-0284-42f3-b591-b9f475fffd03 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:47.142790 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b3ac3-0284-42f3-b591-b9f475fffd03 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:47.147371 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426b6ce-f495-41b4-b150-996789302d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:47.148013 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426b6ce-f495-41b4-b150-996789302d82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:47.148288 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426b6ce-f495-41b4-b150-996789302d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:47.148613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426b6ce-f495-41b4-b150-996789302d82 admin admin] PATH_INFO: `/v3/projects/21e5a87b3b1c4b1fb431752b4e398beb/users/4f663884a36042e2ac0dcc9c94e6c2e2/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:47.163526 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426b6ce-f495-41b4-b150-996789302d82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21e5a87b3b1c4b1fb431752b4e398beb, user_id=4f663884a36042e2ac0dcc9c94e6c2e2, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:47.166635 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426b6ce-f495-41b4-b150-996789302d82 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:47.184610 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1298/2623] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:09:47 2022] PUT /identity/v3/projects/21e5a87b3b1c4b1fb431752b4e398beb/users/4f663884a36042e2ac0dcc9c94e6c2e2/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:47.189154 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496ebc6-48a8-48b3-9036-da0e6b3130f2 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:47.199412 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496ebc6-48a8-48b3-9036-da0e6b3130f2 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:47.260057 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86871b-a796-481f-8f51-055377104bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:47.260710 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86871b-a796-481f-8f51-055377104bcf admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:47.260998 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86871b-a796-481f-8f51-055377104bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:47.261276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86871b-a796-481f-8f51-055377104bcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:47.261967 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc86871b-a796-481f-8f51-055377104bcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:47.263561 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc86871b-a796-481f-8f51-055377104bcf admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:47.273692 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1326/2624] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:09:47.277841 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426b6ce-f495-41b4-b150-996789302d82 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:47.288704 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426b6ce-f495-41b4-b150-996789302d82 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:47.293357 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6980968-f2a0-4f0b-878e-91ba77fe34bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:47.294037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6980968-f2a0-4f0b-878e-91ba77fe34bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:47.294314 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6980968-f2a0-4f0b-878e-91ba77fe34bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:47.294584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6980968-f2a0-4f0b-878e-91ba77fe34bb admin admin] PATH_INFO: `/v3/projects/21e5a87b3b1c4b1fb431752b4e398beb/users/4f663884a36042e2ac0dcc9c94e6c2e2/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:47.296579 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6980968-f2a0-4f0b-878e-91ba77fe34bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=21e5a87b3b1c4b1fb431752b4e398beb, user_id=4f663884a36042e2ac0dcc9c94e6c2e2, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:47.299221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6980968-f2a0-4f0b-878e-91ba77fe34bb admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:47.309118 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1299/2625] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:09:47 2022] PUT /identity/v3/projects/21e5a87b3b1c4b1fb431752b4e398beb/users/4f663884a36042e2ac0dcc9c94e6c2e2/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:09:47.319460 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc4c44-b053-492a-840a-dbbdf266e066 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:47.319795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc4c44-b053-492a-840a-dbbdf266e066 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:47.320063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc4c44-b053-492a-840a-dbbdf266e066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:47.347847 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-eecc4c44-b053-492a-840a-dbbdf266e066 None None] MFA Rules not processed for user `4f663884a36042e2ac0dcc9c94e6c2e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:47.349257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-eecc4c44-b053-492a-840a-dbbdf266e066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:47.415826 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1327/2626] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:47 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:47.427181 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6980968-f2a0-4f0b-878e-91ba77fe34bb admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:47.428953 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f6980968-f2a0-4f0b-878e-91ba77fe34bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:47.492403 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6980968-f2a0-4f0b-878e-91ba77fe34bb admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:47.497559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443285b-459d-4929-af71-4155f0fedd34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:47.498182 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443285b-459d-4929-af71-4155f0fedd34 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:47.498460 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443285b-459d-4929-af71-4155f0fedd34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:47.498734 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443285b-459d-4929-af71-4155f0fedd34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:47.508444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443285b-459d-4929-af71-4155f0fedd34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:47.509871 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443285b-459d-4929-af71-4155f0fedd34 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:47.519693 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1300/2627] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:09:47 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:09:47.559252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc4c44-b053-492a-840a-dbbdf266e066 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:47.560714 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-eecc4c44-b053-492a-840a-dbbdf266e066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:47.572902 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc4c44-b053-492a-840a-dbbdf266e066 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:47.576485 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79dde45-9f02-4c9d-b8ea-acebaa267a4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:47.577092 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79dde45-9f02-4c9d-b8ea-acebaa267a4b service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:47.577374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79dde45-9f02-4c9d-b8ea-acebaa267a4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:47.577644 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79dde45-9f02-4c9d-b8ea-acebaa267a4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:47.621310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:47.635083 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79dde45-9f02-4c9d-b8ea-acebaa267a4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:47.636546 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79dde45-9f02-4c9d-b8ea-acebaa267a4b service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:47.646246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1328/2628] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:09:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:09:54.736122 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443285b-459d-4929-af71-4155f0fedd34 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:54.792728 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:54.826784 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443285b-459d-4929-af71-4155f0fedd34 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:54.832126 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586f4e4-32dc-481e-929d-6a5dea47934f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:54.832789 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586f4e4-32dc-481e-929d-6a5dea47934f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:54.833065 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586f4e4-32dc-481e-929d-6a5dea47934f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:54.833339 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586f4e4-32dc-481e-929d-6a5dea47934f admin admin] PATH_INFO: `/v3/users/4d2eaaca53414894a263418b6917a8ce` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:54.834495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2586f4e4-32dc-481e-929d-6a5dea47934f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d2eaaca53414894a263418b6917a8ce)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:54.836501 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2586f4e4-32dc-481e-929d-6a5dea47934f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:54.879517 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-2586f4e4-32dc-481e-929d-6a5dea47934f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d2eaaca53414894a263418b6917a8ce'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:54.886214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-2586f4e4-32dc-481e-929d-6a5dea47934f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d2eaaca53414894a263418b6917a8ce'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:54.898572 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-2586f4e4-32dc-481e-929d-6a5dea47934f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d2eaaca53414894a263418b6917a8ce'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:54.899255 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-2586f4e4-32dc-481e-929d-6a5dea47934f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d2eaaca53414894a263418b6917a8ce'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:54.899855 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-2586f4e4-32dc-481e-929d-6a5dea47934f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d2eaaca53414894a263418b6917a8ce'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:54.913139 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1301/2629] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:09:54 2022] DELETE /identity/v3/users/4d2eaaca53414894a263418b6917a8ce => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:54.918012 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a12373-4364-4490-96bc-0fb898ad34e4 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:54.918512 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a12373-4364-4490-96bc-0fb898ad34e4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:54.918615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a12373-4364-4490-96bc-0fb898ad34e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:54.938191 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b6a12373-4364-4490-96bc-0fb898ad34e4 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:54.939403 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b6a12373-4364-4490-96bc-0fb898ad34e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:55.005249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1329/2630] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:55.015066 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586f4e4-32dc-481e-929d-6a5dea47934f admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:55.017241 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2586f4e4-32dc-481e-929d-6a5dea47934f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:55.095148 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586f4e4-32dc-481e-929d-6a5dea47934f admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:55.099072 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07fa4b-7daf-4737-90dd-ca94c43af8ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:55.099691 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07fa4b-7daf-4737-90dd-ca94c43af8ed service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:55.099968 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07fa4b-7daf-4737-90dd-ca94c43af8ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:55.100237 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07fa4b-7daf-4737-90dd-ca94c43af8ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:55.109573 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb07fa4b-7daf-4737-90dd-ca94c43af8ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:55.111039 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb07fa4b-7daf-4737-90dd-ca94c43af8ed service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:55.121043 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1302/2631] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:09:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 13:09:55.385937 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a12373-4364-4490-96bc-0fb898ad34e4 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:55.387491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b6a12373-4364-4490-96bc-0fb898ad34e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:55.400432 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a12373-4364-4490-96bc-0fb898ad34e4 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:55.405395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed889e19-e14f-4158-8c53-2f7b2a46365c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:55.405659 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed889e19-e14f-4158-8c53-2f7b2a46365c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:55.405942 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed889e19-e14f-4158-8c53-2f7b2a46365c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:55.406438 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed889e19-e14f-4158-8c53-2f7b2a46365c admin admin] PATH_INFO: `/v3/projects/7482fc21505f4ba7970034e9e6d807b8` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:55.407973 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed889e19-e14f-4158-8c53-2f7b2a46365c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7482fc21505f4ba7970034e9e6d807b8)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:55.410788 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed889e19-e14f-4158-8c53-2f7b2a46365c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:55.456401 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ed889e19-e14f-4158-8c53-2f7b2a46365c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7482fc21505f4ba7970034e9e6d807b8', 'request_id': 'req-ed889e19-e14f-4158-8c53-2f7b2a46365c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:55.461686 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ed889e19-e14f-4158-8c53-2f7b2a46365c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7482fc21505f4ba7970034e9e6d807b8', 'request_id': 'req-ed889e19-e14f-4158-8c53-2f7b2a46365c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:55.468557 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ed889e19-e14f-4158-8c53-2f7b2a46365c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7482fc21505f4ba7970034e9e6d807b8', 'request_id': 'req-ed889e19-e14f-4158-8c53-2f7b2a46365c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:55.479460 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ed889e19-e14f-4158-8c53-2f7b2a46365c admin admin] The token cache is being invalidate because project 7482fc21505f4ba7970034e9e6d807b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:09:55.479922 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ed889e19-e14f-4158-8c53-2f7b2a46365c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:55.480397 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ed889e19-e14f-4158-8c53-2f7b2a46365c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:55.481880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1330/2632] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:09:55 2022] DELETE /identity/v3/projects/7482fc21505f4ba7970034e9e6d807b8 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:55.486444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07fa4b-7daf-4737-90dd-ca94c43af8ed service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:55.495786 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07fa4b-7daf-4737-90dd-ca94c43af8ed service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:55.532352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:55.552606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba139a8-5df4-4a77-a109-eaa6c78a95c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:55.553233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba139a8-5df4-4a77-a109-eaa6c78a95c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:55.553512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba139a8-5df4-4a77-a109-eaa6c78a95c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:55.553783 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba139a8-5df4-4a77-a109-eaa6c78a95c6 admin admin] PATH_INFO: `/v3/users/7d6cf24f00124aa6b23a2c8e63eadd5d` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:55.554923 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba139a8-5df4-4a77-a109-eaa6c78a95c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d6cf24f00124aa6b23a2c8e63eadd5d)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:55.556878 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba139a8-5df4-4a77-a109-eaa6c78a95c6 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:55.605807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8ba139a8-5df4-4a77-a109-eaa6c78a95c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d6cf24f00124aa6b23a2c8e63eadd5d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:55.613114 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8ba139a8-5df4-4a77-a109-eaa6c78a95c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d6cf24f00124aa6b23a2c8e63eadd5d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:55.624728 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8ba139a8-5df4-4a77-a109-eaa6c78a95c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d6cf24f00124aa6b23a2c8e63eadd5d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:55.625394 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8ba139a8-5df4-4a77-a109-eaa6c78a95c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d6cf24f00124aa6b23a2c8e63eadd5d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:55.626006 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8ba139a8-5df4-4a77-a109-eaa6c78a95c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d6cf24f00124aa6b23a2c8e63eadd5d'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:55.638057 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1303/2633] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:09:55 2022] DELETE /identity/v3/users/7d6cf24f00124aa6b23a2c8e63eadd5d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:55.911209 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed889e19-e14f-4158-8c53-2f7b2a46365c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:55.911209 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ed889e19-e14f-4158-8c53-2f7b2a46365c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:55.925465 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed889e19-e14f-4158-8c53-2f7b2a46365c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:55.985887 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c14e6-04ee-45de-8c89-b49c40563056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:55.986526 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c14e6-04ee-45de-8c89-b49c40563056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:55.986795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c14e6-04ee-45de-8c89-b49c40563056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:55.987135 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c14e6-04ee-45de-8c89-b49c40563056 admin admin] PATH_INFO: `/v3/projects/82ff19796ea641b191c4bfa58fa3484e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:55.988237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718c14e6-04ee-45de-8c89-b49c40563056 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82ff19796ea641b191c4bfa58fa3484e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:55.990187 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718c14e6-04ee-45de-8c89-b49c40563056 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:56.001847 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:56.013152 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba139a8-5df4-4a77-a109-eaa6c78a95c6 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:56.014696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8ba139a8-5df4-4a77-a109-eaa6c78a95c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:56.028018 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba139a8-5df4-4a77-a109-eaa6c78a95c6 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:56.041696 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-718c14e6-04ee-45de-8c89-b49c40563056 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82ff19796ea641b191c4bfa58fa3484e', 'request_id': 'req-718c14e6-04ee-45de-8c89-b49c40563056'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.047234 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-718c14e6-04ee-45de-8c89-b49c40563056 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82ff19796ea641b191c4bfa58fa3484e', 'request_id': 'req-718c14e6-04ee-45de-8c89-b49c40563056'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.054528 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-718c14e6-04ee-45de-8c89-b49c40563056 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82ff19796ea641b191c4bfa58fa3484e', 'request_id': 'req-718c14e6-04ee-45de-8c89-b49c40563056'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.065732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-718c14e6-04ee-45de-8c89-b49c40563056 admin admin] The token cache is being invalidate because project 82ff19796ea641b191c4bfa58fa3484e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:09:56.066083 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-718c14e6-04ee-45de-8c89-b49c40563056 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.066772 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-718c14e6-04ee-45de-8c89-b49c40563056 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.068957 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1331/2634] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:09:55 2022] DELETE /identity/v3/projects/82ff19796ea641b191c4bfa58fa3484e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:56.072952 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c14e6-04ee-45de-8c89-b49c40563056 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:56.074272 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-718c14e6-04ee-45de-8c89-b49c40563056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:56.084260 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62915580-7e56-426f-a842-fb3ea3c1542e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:56.084937 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62915580-7e56-426f-a842-fb3ea3c1542e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:56.085246 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62915580-7e56-426f-a842-fb3ea3c1542e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:56.085522 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62915580-7e56-426f-a842-fb3ea3c1542e admin admin] PATH_INFO: `/v3/users/5953fa87df04465d9867fa284084ccfe` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:56.086692 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62915580-7e56-426f-a842-fb3ea3c1542e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5953fa87df04465d9867fa284084ccfe)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:56.088640 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62915580-7e56-426f-a842-fb3ea3c1542e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:56.140916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-62915580-7e56-426f-a842-fb3ea3c1542e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5953fa87df04465d9867fa284084ccfe'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.148097 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-62915580-7e56-426f-a842-fb3ea3c1542e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5953fa87df04465d9867fa284084ccfe'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.151403 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c14e6-04ee-45de-8c89-b49c40563056 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:56.152671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:56.157065 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977bfeb7-e69f-47aa-8f67-3f48c46b4715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:56.157710 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977bfeb7-e69f-47aa-8f67-3f48c46b4715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:56.157997 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977bfeb7-e69f-47aa-8f67-3f48c46b4715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:56.158269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977bfeb7-e69f-47aa-8f67-3f48c46b4715 admin admin] PATH_INFO: `/v3/users/4f663884a36042e2ac0dcc9c94e6c2e2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:56.159475 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977bfeb7-e69f-47aa-8f67-3f48c46b4715 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f663884a36042e2ac0dcc9c94e6c2e2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:56.161764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977bfeb7-e69f-47aa-8f67-3f48c46b4715 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:56.162769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-62915580-7e56-426f-a842-fb3ea3c1542e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5953fa87df04465d9867fa284084ccfe'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.163601 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-62915580-7e56-426f-a842-fb3ea3c1542e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5953fa87df04465d9867fa284084ccfe'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.164612 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-62915580-7e56-426f-a842-fb3ea3c1542e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5953fa87df04465d9867fa284084ccfe'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.177934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1304/2635] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:09:56 2022] DELETE /identity/v3/users/5953fa87df04465d9867fa284084ccfe => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:09:56.182445 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc4cd7-a332-4ea0-9227-80064ff808fc None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:56.182732 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc4cd7-a332-4ea0-9227-80064ff808fc None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:56.183000 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc4cd7-a332-4ea0-9227-80064ff808fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:56.201673 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-36bc4cd7-a332-4ea0-9227-80064ff808fc None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:56.203635 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-36bc4cd7-a332-4ea0-9227-80064ff808fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:56.207229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-977bfeb7-e69f-47aa-8f67-3f48c46b4715 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f663884a36042e2ac0dcc9c94e6c2e2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.219614 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-977bfeb7-e69f-47aa-8f67-3f48c46b4715 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f663884a36042e2ac0dcc9c94e6c2e2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.220288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-977bfeb7-e69f-47aa-8f67-3f48c46b4715 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f663884a36042e2ac0dcc9c94e6c2e2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.220931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-977bfeb7-e69f-47aa-8f67-3f48c46b4715 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f663884a36042e2ac0dcc9c94e6c2e2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.228819 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-977bfeb7-e69f-47aa-8f67-3f48c46b4715 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f663884a36042e2ac0dcc9c94e6c2e2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.241110 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1332/2636] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:09:56 2022] DELETE /identity/v3/users/4f663884a36042e2ac0dcc9c94e6c2e2 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:09:56.264973 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1305/2637] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 13:09:56.278637 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977bfeb7-e69f-47aa-8f67-3f48c46b4715 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:56.280078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-977bfeb7-e69f-47aa-8f67-3f48c46b4715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:56.300650 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:56.342512 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977bfeb7-e69f-47aa-8f67-3f48c46b4715 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:56.346460 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b520a-23a0-4dfd-95ae-a8db20b3aa1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:56.346961 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b520a-23a0-4dfd-95ae-a8db20b3aa1b service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:56.347713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b520a-23a0-4dfd-95ae-a8db20b3aa1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:56.347713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b520a-23a0-4dfd-95ae-a8db20b3aa1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:56.413986 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b520a-23a0-4dfd-95ae-a8db20b3aa1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:56.415421 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b520a-23a0-4dfd-95ae-a8db20b3aa1b service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:56.425953 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1333/2638] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:09:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:09:56.521262 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc4cd7-a332-4ea0-9227-80064ff808fc None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:56.522853 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-36bc4cd7-a332-4ea0-9227-80064ff808fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:56.535950 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc4cd7-a332-4ea0-9227-80064ff808fc None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:56.541338 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4ae13-623c-4dce-89cd-7e6a1cba3088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:56.542037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4ae13-623c-4dce-89cd-7e6a1cba3088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:56.542281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4ae13-623c-4dce-89cd-7e6a1cba3088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:56.542554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4ae13-623c-4dce-89cd-7e6a1cba3088 admin admin] PATH_INFO: `/v3/projects/21e5a87b3b1c4b1fb431752b4e398beb` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:56.543992 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e4ae13-623c-4dce-89cd-7e6a1cba3088 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21e5a87b3b1c4b1fb431752b4e398beb)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:56.546336 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e4ae13-623c-4dce-89cd-7e6a1cba3088 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:56.599037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-61e4ae13-623c-4dce-89cd-7e6a1cba3088 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21e5a87b3b1c4b1fb431752b4e398beb', 'request_id': 'req-61e4ae13-623c-4dce-89cd-7e6a1cba3088'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.604751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-61e4ae13-623c-4dce-89cd-7e6a1cba3088 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21e5a87b3b1c4b1fb431752b4e398beb', 'request_id': 'req-61e4ae13-623c-4dce-89cd-7e6a1cba3088'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.612770 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-61e4ae13-623c-4dce-89cd-7e6a1cba3088 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21e5a87b3b1c4b1fb431752b4e398beb', 'request_id': 'req-61e4ae13-623c-4dce-89cd-7e6a1cba3088'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.624999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-61e4ae13-623c-4dce-89cd-7e6a1cba3088 admin admin] The token cache is being invalidate because project 21e5a87b3b1c4b1fb431752b4e398beb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:09:56.625353 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-61e4ae13-623c-4dce-89cd-7e6a1cba3088 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.626241 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-61e4ae13-623c-4dce-89cd-7e6a1cba3088 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.627800 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1306/2639] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:09:56 2022] DELETE /identity/v3/projects/21e5a87b3b1c4b1fb431752b4e398beb => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:56.633895 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec93d6-c90c-4353-ba70-a176c0249bb3 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:56.634236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec93d6-c90c-4353-ba70-a176c0249bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:56.634514 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec93d6-c90c-4353-ba70-a176c0249bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:56.654101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e3ec93d6-c90c-4353-ba70-a176c0249bb3 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:56.655330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e3ec93d6-c90c-4353-ba70-a176c0249bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:56.701154 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4ae13-623c-4dce-89cd-7e6a1cba3088 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:56.702593 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-61e4ae13-623c-4dce-89cd-7e6a1cba3088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:56.741340 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1334/2640] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:56.749688 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274e6aa-25f5-46c8-a708-ab04654d41e1 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:56.749981 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274e6aa-25f5-46c8-a708-ab04654d41e1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:56.750397 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274e6aa-25f5-46c8-a708-ab04654d41e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:56.769882 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6274e6aa-25f5-46c8-a708-ab04654d41e1 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:56.771121 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6274e6aa-25f5-46c8-a708-ab04654d41e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:56.776400 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4ae13-623c-4dce-89cd-7e6a1cba3088 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:56.779195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1335/2641] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 13:09:56.781538 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fbf27e-a4c6-4e4a-97a7-8323f673c2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:56.782175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbf27e-a4c6-4e4a-97a7-8323f673c2b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:56.782453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbf27e-a4c6-4e4a-97a7-8323f673c2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:56.782954 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbf27e-a4c6-4e4a-97a7-8323f673c2b1 admin admin] PATH_INFO: `/v3/projects/6bd7234940c949038d532a9a0ac3c2bc` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:56.783750 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274e6aa-25f5-46c8-a708-ab04654d41e1 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:56.783885 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fbf27e-a4c6-4e4a-97a7-8323f673c2b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bd7234940c949038d532a9a0ac3c2bc)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:56.785809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fbf27e-a4c6-4e4a-97a7-8323f673c2b1 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:56.788151 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:56.793017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274e6aa-25f5-46c8-a708-ab04654d41e1 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:56.798194 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa614d67-9de5-404b-af28-1d690e7a31c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:56.798796 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa614d67-9de5-404b-af28-1d690e7a31c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:56.799273 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa614d67-9de5-404b-af28-1d690e7a31c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:56.799386 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa614d67-9de5-404b-af28-1d690e7a31c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:56.800017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa614d67-9de5-404b-af28-1d690e7a31c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:09:56.800343 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa614d67-9de5-404b-af28-1d690e7a31c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:56.801911 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa614d67-9de5-404b-af28-1d690e7a31c8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:56.814759 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1336/2642] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 13:09:56.819174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa614d67-9de5-404b-af28-1d690e7a31c8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:56.830378 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa614d67-9de5-404b-af28-1d690e7a31c8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:56.835092 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-59fbf27e-a4c6-4e4a-97a7-8323f673c2b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bd7234940c949038d532a9a0ac3c2bc', 'request_id': 'req-59fbf27e-a4c6-4e4a-97a7-8323f673c2b1'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.840763 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-59fbf27e-a4c6-4e4a-97a7-8323f673c2b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bd7234940c949038d532a9a0ac3c2bc', 'request_id': 'req-59fbf27e-a4c6-4e4a-97a7-8323f673c2b1'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.850520 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-59fbf27e-a4c6-4e4a-97a7-8323f673c2b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bd7234940c949038d532a9a0ac3c2bc', 'request_id': 'req-59fbf27e-a4c6-4e4a-97a7-8323f673c2b1'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.861755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-59fbf27e-a4c6-4e4a-97a7-8323f673c2b1 admin admin] The token cache is being invalidate because project 6bd7234940c949038d532a9a0ac3c2bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:09:56.862044 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-59fbf27e-a4c6-4e4a-97a7-8323f673c2b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.862693 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-59fbf27e-a4c6-4e4a-97a7-8323f673c2b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:56.864196 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1307/2643] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:09:56 2022] DELETE /identity/v3/projects/6bd7234940c949038d532a9a0ac3c2bc => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:56.868130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fbf27e-a4c6-4e4a-97a7-8323f673c2b1 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:56.869438 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-59fbf27e-a4c6-4e4a-97a7-8323f673c2b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:56.879167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:56.891934 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a9559-ca3f-42a3-bbeb-9d68acd6684d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:56.892567 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a9559-ca3f-42a3-bbeb-9d68acd6684d admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:56.892849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a9559-ca3f-42a3-bbeb-9d68acd6684d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:56.893146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a9559-ca3f-42a3-bbeb-9d68acd6684d admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:56.893857 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a9559-ca3f-42a3-bbeb-9d68acd6684d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:56.895665 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a9559-ca3f-42a3-bbeb-9d68acd6684d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:56.926632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1337/2644] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:09:56 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:09:56.929314 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a9559-ca3f-42a3-bbeb-9d68acd6684d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:56.930954 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-065a9559-ca3f-42a3-bbeb-9d68acd6684d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:56.935825 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fbf27e-a4c6-4e4a-97a7-8323f673c2b1 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:56.990603 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b24a8-9a97-4801-93da-63edb16a4fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:56.991241 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b24a8-9a97-4801-93da-63edb16a4fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:56.991523 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b24a8-9a97-4801-93da-63edb16a4fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:56.991795 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b24a8-9a97-4801-93da-63edb16a4fb6 admin admin] PATH_INFO: `/v3/users/08a7884cded14138b23e9142cc8e1021` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:56.993087 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b24a8-9a97-4801-93da-63edb16a4fb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08a7884cded14138b23e9142cc8e1021)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:56.995029 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b24a8-9a97-4801-93da-63edb16a4fb6 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:57.002097 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a9559-ca3f-42a3-bbeb-9d68acd6684d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:57.006864 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9001df85-6f02-4182-8f5e-d687a4784623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:57.007613 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9001df85-6f02-4182-8f5e-d687a4784623 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:57.007891 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9001df85-6f02-4182-8f5e-d687a4784623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:57.008160 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9001df85-6f02-4182-8f5e-d687a4784623 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:57.008851 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9001df85-6f02-4182-8f5e-d687a4784623 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:57.010772 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9001df85-6f02-4182-8f5e-d687a4784623 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:57.034832 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1338/2645] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:09:56 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:09:57.038938 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9001df85-6f02-4182-8f5e-d687a4784623 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:57.042700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:57.048364 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-180b24a8-9a97-4801-93da-63edb16a4fb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08a7884cded14138b23e9142cc8e1021'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:57.049030 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9001df85-6f02-4182-8f5e-d687a4784623 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:57.054640 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f10c18-206e-4adc-b7f8-c6fb4bcdff50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:57.055237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f10c18-206e-4adc-b7f8-c6fb4bcdff50 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:57.055519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f10c18-206e-4adc-b7f8-c6fb4bcdff50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:57.055790 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f10c18-206e-4adc-b7f8-c6fb4bcdff50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:57.056162 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-180b24a8-9a97-4801-93da-63edb16a4fb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08a7884cded14138b23e9142cc8e1021'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:57.056516 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f10c18-206e-4adc-b7f8-c6fb4bcdff50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:57.058151 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f10c18-206e-4adc-b7f8-c6fb4bcdff50 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:57.069135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-180b24a8-9a97-4801-93da-63edb16a4fb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08a7884cded14138b23e9142cc8e1021'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:57.072589 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1339/2646] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 13:09:57.072732 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-180b24a8-9a97-4801-93da-63edb16a4fb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08a7884cded14138b23e9142cc8e1021'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:57.072732 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-180b24a8-9a97-4801-93da-63edb16a4fb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08a7884cded14138b23e9142cc8e1021'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:57.073510 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f10c18-206e-4adc-b7f8-c6fb4bcdff50 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:57.075334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-89f10c18-206e-4adc-b7f8-c6fb4bcdff50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:57.084026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1308/2647] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:09:56 2022] DELETE /identity/v3/users/08a7884cded14138b23e9142cc8e1021 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:09:57.091602 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f10c18-206e-4adc-b7f8-c6fb4bcdff50 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:57.152689 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2bc583-f98f-45d6-8772-55dbdbb2ffeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:57.153365 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bc583-f98f-45d6-8772-55dbdbb2ffeb admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:57.153650 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bc583-f98f-45d6-8772-55dbdbb2ffeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:57.153927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bc583-f98f-45d6-8772-55dbdbb2ffeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:57.154660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2bc583-f98f-45d6-8772-55dbdbb2ffeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:57.156270 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2bc583-f98f-45d6-8772-55dbdbb2ffeb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:57.167149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1340/2648] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:09:57.171664 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b24a8-9a97-4801-93da-63edb16a4fb6 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:57.183786 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b24a8-9a97-4801-93da-63edb16a4fb6 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:57.190167 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c53ffb-3d4a-4a81-84c3-0ec4dbf31769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:57.190840 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c53ffb-3d4a-4a81-84c3-0ec4dbf31769 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:57.191122 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c53ffb-3d4a-4a81-84c3-0ec4dbf31769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:57.191399 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c53ffb-3d4a-4a81-84c3-0ec4dbf31769 admin admin] PATH_INFO: `/v3/projects/0e1fd4032187401696c16097246be4da/users/3ffa2a2bfcbd4e819139418c0f35fb75/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:57.219199 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c53ffb-3d4a-4a81-84c3-0ec4dbf31769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e1fd4032187401696c16097246be4da, user_id=3ffa2a2bfcbd4e819139418c0f35fb75, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:57.222077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c53ffb-3d4a-4a81-84c3-0ec4dbf31769 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:57.239407 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1309/2649] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:09:57 2022] PUT /identity/v3/projects/0e1fd4032187401696c16097246be4da/users/3ffa2a2bfcbd4e819139418c0f35fb75/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:57.252905 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c6f2c-fc5a-457b-9b9f-74ec9753f0f0 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:57.253196 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c6f2c-fc5a-457b-9b9f-74ec9753f0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:57.254007 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c6f2c-fc5a-457b-9b9f-74ec9753f0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:57.282675 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c45c6f2c-fc5a-457b-9b9f-74ec9753f0f0 None None] MFA Rules not processed for user `3ffa2a2bfcbd4e819139418c0f35fb75`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:57.283886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c45c6f2c-fc5a-457b-9b9f-74ec9753f0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:57.322140 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:57.350951 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1341/2650] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:57.355266 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c53ffb-3d4a-4a81-84c3-0ec4dbf31769 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:57.365962 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c6f2c-fc5a-457b-9b9f-74ec9753f0f0 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:57.367489 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c45c6f2c-fc5a-457b-9b9f-74ec9753f0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:57.421152 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c53ffb-3d4a-4a81-84c3-0ec4dbf31769 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:57.426884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657cc83-deb9-4f1f-96b4-fbfa5a3492e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:57.427502 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657cc83-deb9-4f1f-96b4-fbfa5a3492e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:57.427783 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657cc83-deb9-4f1f-96b4-fbfa5a3492e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:57.428061 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657cc83-deb9-4f1f-96b4-fbfa5a3492e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:57.428767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657cc83-deb9-4f1f-96b4-fbfa5a3492e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:57.430662 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657cc83-deb9-4f1f-96b4-fbfa5a3492e5 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:57.439347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c6f2c-fc5a-457b-9b9f-74ec9753f0f0 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:57.443932 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf672e-2497-4207-b785-bfffb0aff2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:57.444623 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf672e-2497-4207-b785-bfffb0aff2b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:57.444901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf672e-2497-4207-b785-bfffb0aff2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:57.445174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf672e-2497-4207-b785-bfffb0aff2b4 admin admin] PATH_INFO: `/v3/projects/a45de26351054ec28ba0e43ee9a70a3d` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:57.446341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf672e-2497-4207-b785-bfffb0aff2b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a45de26351054ec28ba0e43ee9a70a3d)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:57.448253 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf672e-2497-4207-b785-bfffb0aff2b4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:57.458887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1310/2651] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:09:57 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:09:57.462530 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657cc83-deb9-4f1f-96b4-fbfa5a3492e5 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:57.497745 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9abf672e-2497-4207-b785-bfffb0aff2b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a45de26351054ec28ba0e43ee9a70a3d', 'request_id': 'req-9abf672e-2497-4207-b785-bfffb0aff2b4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:57.504201 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9abf672e-2497-4207-b785-bfffb0aff2b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a45de26351054ec28ba0e43ee9a70a3d', 'request_id': 'req-9abf672e-2497-4207-b785-bfffb0aff2b4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:57.512320 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9abf672e-2497-4207-b785-bfffb0aff2b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a45de26351054ec28ba0e43ee9a70a3d', 'request_id': 'req-9abf672e-2497-4207-b785-bfffb0aff2b4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:57.523285 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9abf672e-2497-4207-b785-bfffb0aff2b4 admin admin] The token cache is being invalidate because project a45de26351054ec28ba0e43ee9a70a3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:09:57.523576 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9abf672e-2497-4207-b785-bfffb0aff2b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:57.524209 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657cc83-deb9-4f1f-96b4-fbfa5a3492e5 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:57.524350 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9abf672e-2497-4207-b785-bfffb0aff2b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:09:57.525862 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1342/2652] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:09:57 2022] DELETE /identity/v3/projects/a45de26351054ec28ba0e43ee9a70a3d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:09:57.530416 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dc603-b8b5-4180-b160-3d6ba2061afe None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:57.530700 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dc603-b8b5-4180-b160-3d6ba2061afe None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:57.531010 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dc603-b8b5-4180-b160-3d6ba2061afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:57.549385 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:57.550567 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ab0dc603-b8b5-4180-b160-3d6ba2061afe None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:57.551778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ab0dc603-b8b5-4180-b160-3d6ba2061afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:57.582955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10251d-4ec5-439b-84ca-9758a1751908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:57.583558 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10251d-4ec5-439b-84ca-9758a1751908 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:57.583832 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10251d-4ec5-439b-84ca-9758a1751908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:57.584099 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10251d-4ec5-439b-84ca-9758a1751908 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:57.584785 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd10251d-4ec5-439b-84ca-9758a1751908 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:57.586688 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd10251d-4ec5-439b-84ca-9758a1751908 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:57.607830 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1311/2653] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:09:57 2022] POST /identity/v3/users => generated 290 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:09:57.617808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1343/2654] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 13:09:57.617956 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10251d-4ec5-439b-84ca-9758a1751908 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:57.617956 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-dd10251d-4ec5-439b-84ca-9758a1751908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:57.619237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95cb808-8b53-431d-8c80-5e48be95e050 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:57.619519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95cb808-8b53-431d-8c80-5e48be95e050 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:57.619790 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95cb808-8b53-431d-8c80-5e48be95e050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:57.626182 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10251d-4ec5-439b-84ca-9758a1751908 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:57.631397 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c961141-c147-4d5d-86af-06d4d89df042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:57.632000 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c961141-c147-4d5d-86af-06d4d89df042 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:57.632277 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c961141-c147-4d5d-86af-06d4d89df042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:57.632582 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c961141-c147-4d5d-86af-06d4d89df042 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:57.633267 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c961141-c147-4d5d-86af-06d4d89df042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:57.634861 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c961141-c147-4d5d-86af-06d4d89df042 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:57.640476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c95cb808-8b53-431d-8c80-5e48be95e050 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:57.641670 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c95cb808-8b53-431d-8c80-5e48be95e050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:57.645184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1312/2655] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 13:09:57.649505 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c961141-c147-4d5d-86af-06d4d89df042 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:57.653362 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1344/2656] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 13:09:57.657616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95cb808-8b53-431d-8c80-5e48be95e050 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:57.659427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c961141-c147-4d5d-86af-06d4d89df042 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:57.662354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:57.665039 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee46b2-d097-4a3d-998d-d195d97d446b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:57.665039 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee46b2-d097-4a3d-998d-d195d97d446b admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:57.665264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee46b2-d097-4a3d-998d-d195d97d446b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:57.665412 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee46b2-d097-4a3d-998d-d195d97d446b admin admin] PATH_INFO: `/v3/projects/8b82b1a86e2c494588b140bd68e8be50/users/33752a1716fa494488c689fd2343ebb5/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:57.669166 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95cb808-8b53-431d-8c80-5e48be95e050 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:57.676883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecd1fe-47d0-442d-bde2-a5f9fbc0ba81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:57.677675 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecd1fe-47d0-442d-bde2-a5f9fbc0ba81 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:57.677963 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecd1fe-47d0-442d-bde2-a5f9fbc0ba81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:57.678267 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecd1fe-47d0-442d-bde2-a5f9fbc0ba81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:57.678966 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acecd1fe-47d0-442d-bde2-a5f9fbc0ba81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:09:57.679268 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acecd1fe-47d0-442d-bde2-a5f9fbc0ba81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:57.680905 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acecd1fe-47d0-442d-bde2-a5f9fbc0ba81 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:57.682207 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee46b2-d097-4a3d-998d-d195d97d446b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b82b1a86e2c494588b140bd68e8be50, user_id=33752a1716fa494488c689fd2343ebb5, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:57.684922 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee46b2-d097-4a3d-998d-d195d97d446b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:57.693647 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1345/2657] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 13:09:57.697868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecd1fe-47d0-442d-bde2-a5f9fbc0ba81 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:57.698474 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1313/2658] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:09:57 2022] PUT /identity/v3/projects/8b82b1a86e2c494588b140bd68e8be50/users/33752a1716fa494488c689fd2343ebb5/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:57.702181 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee46b2-d097-4a3d-998d-d195d97d446b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:57.707543 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecd1fe-47d0-442d-bde2-a5f9fbc0ba81 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:57.766395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee46b2-d097-4a3d-998d-d195d97d446b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:57.771301 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1633d-3b6f-4a9b-9e3e-c8e047deace0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:57.771934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1633d-3b6f-4a9b-9e3e-c8e047deace0 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:57.772211 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1633d-3b6f-4a9b-9e3e-c8e047deace0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:57.772556 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1633d-3b6f-4a9b-9e3e-c8e047deace0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:57.773259 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1633d-3b6f-4a9b-9e3e-c8e047deace0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:57.775077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1633d-3b6f-4a9b-9e3e-c8e047deace0 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:57.786021 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1314/2659] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 13:09:57.790413 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1633d-3b6f-4a9b-9e3e-c8e047deace0 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:57.801235 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1633d-3b6f-4a9b-9e3e-c8e047deace0 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:57.805971 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b0ec9-b648-41eb-84f9-48bf1803597c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:57.806643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b0ec9-b648-41eb-84f9-48bf1803597c admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:57.806918 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b0ec9-b648-41eb-84f9-48bf1803597c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:57.807192 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b0ec9-b648-41eb-84f9-48bf1803597c admin admin] PATH_INFO: `/v3/projects/8b82b1a86e2c494588b140bd68e8be50/users/33752a1716fa494488c689fd2343ebb5/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:57.809201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b0ec9-b648-41eb-84f9-48bf1803597c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b82b1a86e2c494588b140bd68e8be50, user_id=33752a1716fa494488c689fd2343ebb5, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:57.811861 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b0ec9-b648-41eb-84f9-48bf1803597c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:57.824095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1315/2660] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:09:57 2022] PUT /identity/v3/projects/8b82b1a86e2c494588b140bd68e8be50/users/33752a1716fa494488c689fd2343ebb5/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:57.835771 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f38598-3559-4a5a-b70e-ee81d1ab87cb None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:57.836050 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f38598-3559-4a5a-b70e-ee81d1ab87cb None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:57.836495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f38598-3559-4a5a-b70e-ee81d1ab87cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:57.867270 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-85f38598-3559-4a5a-b70e-ee81d1ab87cb None None] MFA Rules not processed for user `33752a1716fa494488c689fd2343ebb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:57.868554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-85f38598-3559-4a5a-b70e-ee81d1ab87cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:57.892365 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b876339-2e74-47e9-80e0-d8d048f5150e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:57.893009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b876339-2e74-47e9-80e0-d8d048f5150e admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:57.893286 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b876339-2e74-47e9-80e0-d8d048f5150e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:57.893555 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b876339-2e74-47e9-80e0-d8d048f5150e admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:57.894201 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b876339-2e74-47e9-80e0-d8d048f5150e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:57.896093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b876339-2e74-47e9-80e0-d8d048f5150e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:57.920230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:57.928097 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1346/2661] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:09:57 2022] POST /identity/v3/projects => generated 407 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:09:57.929243 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b876339-2e74-47e9-80e0-d8d048f5150e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:57.945298 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1316/2662] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:57 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 13:09:57.958707 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f38598-3559-4a5a-b70e-ee81d1ab87cb None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:57.960755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-85f38598-3559-4a5a-b70e-ee81d1ab87cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:57.998898 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b876339-2e74-47e9-80e0-d8d048f5150e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:58.004688 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08bcc5-d37b-4ca2-ac66-4f742307b4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:58.005429 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08bcc5-d37b-4ca2-ac66-4f742307b4d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:58.005716 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08bcc5-d37b-4ca2-ac66-4f742307b4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:58.005990 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08bcc5-d37b-4ca2-ac66-4f742307b4d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:58.006686 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f08bcc5-d37b-4ca2-ac66-4f742307b4d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:58.008819 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f08bcc5-d37b-4ca2-ac66-4f742307b4d4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:58.025284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f38598-3559-4a5a-b70e-ee81d1ab87cb None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:58.031493 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1347/2663] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:09:57 2022] POST /identity/v3/users => generated 304 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:09:58.035482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08bcc5-d37b-4ca2-ac66-4f742307b4d4 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:58.044686 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08bcc5-d37b-4ca2-ac66-4f742307b4d4 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:58.049843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d07d6d-d7b1-47fa-8c40-278a6942cde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:58.050426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d07d6d-d7b1-47fa-8c40-278a6942cde1 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:58.050705 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d07d6d-d7b1-47fa-8c40-278a6942cde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:58.050976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d07d6d-d7b1-47fa-8c40-278a6942cde1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:58.051657 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d07d6d-d7b1-47fa-8c40-278a6942cde1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:58.053247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d07d6d-d7b1-47fa-8c40-278a6942cde1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:58.064263 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1348/2664] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:09:58.068006 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d07d6d-d7b1-47fa-8c40-278a6942cde1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:58.073815 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda495cd-52de-412b-a04e-ff874c7046a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:58.074433 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda495cd-52de-412b-a04e-ff874c7046a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:58.074713 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda495cd-52de-412b-a04e-ff874c7046a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:58.075021 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda495cd-52de-412b-a04e-ff874c7046a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:58.077445 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d07d6d-d7b1-47fa-8c40-278a6942cde1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:58.082001 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ddedb-8988-4c14-818b-8f75683c4db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:58.082579 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ddedb-8988-4c14-818b-8f75683c4db6 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:58.082860 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ddedb-8988-4c14-818b-8f75683c4db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:58.083129 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ddedb-8988-4c14-818b-8f75683c4db6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:58.083808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ddedb-8988-4c14-818b-8f75683c4db6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:58.085468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ddedb-8988-4c14-818b-8f75683c4db6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:58.096220 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1349/2665] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 13:09:58.100439 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ddedb-8988-4c14-818b-8f75683c4db6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:58.110195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ddedb-8988-4c14-818b-8f75683c4db6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:58.114728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e179e06-a711-4285-a13d-517778feb0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:58.115348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e179e06-a711-4285-a13d-517778feb0d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:58.115626 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e179e06-a711-4285-a13d-517778feb0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:58.115900 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e179e06-a711-4285-a13d-517778feb0d5 admin admin] PATH_INFO: `/v3/projects/32ddc615629d43c2956984b1a2f58a04/users/b461b4df4c62496385b9a059a83a6acb/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:58.130307 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e179e06-a711-4285-a13d-517778feb0d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32ddc615629d43c2956984b1a2f58a04, user_id=b461b4df4c62496385b9a059a83a6acb, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:58.133035 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e179e06-a711-4285-a13d-517778feb0d5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:58.137182 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda495cd-52de-412b-a04e-ff874c7046a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:58.138623 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda495cd-52de-412b-a04e-ff874c7046a4 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:58.147185 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1350/2666] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:09:58 2022] PUT /identity/v3/projects/32ddc615629d43c2956984b1a2f58a04/users/b461b4df4c62496385b9a059a83a6acb/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:09:58.148586 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1317/2667] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:09:57 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 13:09:58.159023 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ae221-9e57-4cd1-91ed-f72b4911c8ac None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:58.159358 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ae221-9e57-4cd1-91ed-f72b4911c8ac None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:58.159637 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ae221-9e57-4cd1-91ed-f72b4911c8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:58.188146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c82ae221-9e57-4cd1-91ed-f72b4911c8ac None None] MFA Rules not processed for user `b461b4df4c62496385b9a059a83a6acb`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:58.191740 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c82ae221-9e57-4cd1-91ed-f72b4911c8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:58.197188 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:58.244626 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda495cd-52de-412b-a04e-ff874c7046a4 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:58.247231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-cda495cd-52de-412b-a04e-ff874c7046a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:58.262802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda495cd-52de-412b-a04e-ff874c7046a4 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:58.279025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1351/2668] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:58 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:58.284258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ae221-9e57-4cd1-91ed-f72b4911c8ac None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:58.316336 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4342793-e20c-42d7-9ba8-6054bd14b1ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:58.316937 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4342793-e20c-42d7-9ba8-6054bd14b1ba service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:58.317215 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4342793-e20c-42d7-9ba8-6054bd14b1ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:58.317486 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4342793-e20c-42d7-9ba8-6054bd14b1ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:58.319277 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a4342793-e20c-42d7-9ba8-6054bd14b1ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:58.329393 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:58.345576 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ae221-9e57-4cd1-91ed-f72b4911c8ac None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:58.350564 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d9c31-51b3-4eb6-b89a-4434372fada9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:58.351173 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d9c31-51b3-4eb6-b89a-4434372fada9 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:58.351447 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d9c31-51b3-4eb6-b89a-4434372fada9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:58.351712 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d9c31-51b3-4eb6-b89a-4434372fada9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:58.352441 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d9c31-51b3-4eb6-b89a-4434372fada9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:58.354316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d9c31-51b3-4eb6-b89a-4434372fada9 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:58.379564 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4342793-e20c-42d7-9ba8-6054bd14b1ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:58.380616 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1352/2669] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:09:58 2022] POST /identity/v3/projects => generated 409 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:09:58.381039 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4342793-e20c-42d7-9ba8-6054bd14b1ba service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:58.384553 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d9c31-51b3-4eb6-b89a-4434372fada9 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:58.390698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1318/2670] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:09:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:09:58.446570 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d9c31-51b3-4eb6-b89a-4434372fada9 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:58.451171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a147a5b4-fcbf-4dda-9ffb-ad9b21d38f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:58.451784 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147a5b4-fcbf-4dda-9ffb-ad9b21d38f7f admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:58.452069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147a5b4-fcbf-4dda-9ffb-ad9b21d38f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:58.452377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147a5b4-fcbf-4dda-9ffb-ad9b21d38f7f admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:58.453564 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a147a5b4-fcbf-4dda-9ffb-ad9b21d38f7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:58.455463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a147a5b4-fcbf-4dda-9ffb-ad9b21d38f7f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:58.477524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1353/2671] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:09:58 2022] POST /identity/v3/users => generated 311 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:09:58.481555 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4342793-e20c-42d7-9ba8-6054bd14b1ba service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:58.492459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4342793-e20c-42d7-9ba8-6054bd14b1ba service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:58.497139 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb81e8-db45-42a0-a808-e1e26a56ed43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:58.497742 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb81e8-db45-42a0-a808-e1e26a56ed43 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:58.498560 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb81e8-db45-42a0-a808-e1e26a56ed43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:58.498560 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb81e8-db45-42a0-a808-e1e26a56ed43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:58.498997 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb81e8-db45-42a0-a808-e1e26a56ed43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:58.500663 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb81e8-db45-42a0-a808-e1e26a56ed43 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:58.515674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1319/2672] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:09:58.517012 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a147a5b4-fcbf-4dda-9ffb-ad9b21d38f7f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:58.526940 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a147a5b4-fcbf-4dda-9ffb-ad9b21d38f7f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:58.532318 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97054dc-00ce-4115-9ac5-4d012fb09401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:58.533120 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97054dc-00ce-4115-9ac5-4d012fb09401 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:58.533300 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97054dc-00ce-4115-9ac5-4d012fb09401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:58.533679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97054dc-00ce-4115-9ac5-4d012fb09401 admin admin] PATH_INFO: `/v3/projects/a1b230e611764afeaf25698ec1efe047/users/df08c90271404a17b647e4e30e705d76/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:58.549873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97054dc-00ce-4115-9ac5-4d012fb09401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b230e611764afeaf25698ec1efe047, user_id=df08c90271404a17b647e4e30e705d76, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:58.552554 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97054dc-00ce-4115-9ac5-4d012fb09401 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:58.560368 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:58.567234 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1354/2673] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:09:58 2022] PUT /identity/v3/projects/a1b230e611764afeaf25698ec1efe047/users/df08c90271404a17b647e4e30e705d76/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:09:58.571606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb81e8-db45-42a0-a808-e1e26a56ed43 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:58.581886 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb81e8-db45-42a0-a808-e1e26a56ed43 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:58.642992 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c7ee9d-7c46-45f6-ac5e-7992d3d05a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:58.643681 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c7ee9d-7c46-45f6-ac5e-7992d3d05a24 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:58.644039 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c7ee9d-7c46-45f6-ac5e-7992d3d05a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:58.644423 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c7ee9d-7c46-45f6-ac5e-7992d3d05a24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:58.645199 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c7ee9d-7c46-45f6-ac5e-7992d3d05a24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:58.646902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c7ee9d-7c46-45f6-ac5e-7992d3d05a24 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:58.658865 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1320/2674] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:09:58.663225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97054dc-00ce-4115-9ac5-4d012fb09401 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:58.677506 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97054dc-00ce-4115-9ac5-4d012fb09401 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:58.684635 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b35a4-00ed-4f97-877e-481e34dda5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:58.685549 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b35a4-00ed-4f97-877e-481e34dda5e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:58.686333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b35a4-00ed-4f97-877e-481e34dda5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:58.686978 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b35a4-00ed-4f97-877e-481e34dda5e2 admin admin] PATH_INFO: `/v3/projects/a1b230e611764afeaf25698ec1efe047/users/df08c90271404a17b647e4e30e705d76/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:58.689648 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b35a4-00ed-4f97-877e-481e34dda5e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b230e611764afeaf25698ec1efe047, user_id=df08c90271404a17b647e4e30e705d76, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:58.692264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b35a4-00ed-4f97-877e-481e34dda5e2 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:58.703708 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1355/2675] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:09:58 2022] PUT /identity/v3/projects/a1b230e611764afeaf25698ec1efe047/users/df08c90271404a17b647e4e30e705d76/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:09:58.713841 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f99268-7e73-4a88-b098-77a35ff1f104 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:58.714206 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f99268-7e73-4a88-b098-77a35ff1f104 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:58.714545 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f99268-7e73-4a88-b098-77a35ff1f104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:58.744418 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-03f99268-7e73-4a88-b098-77a35ff1f104 None None] MFA Rules not processed for user `df08c90271404a17b647e4e30e705d76`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:09:58.745588 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-03f99268-7e73-4a88-b098-77a35ff1f104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:09:58.811030 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1321/2676] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:09:58 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:09:58.822532 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b35a4-00ed-4f97-877e-481e34dda5e2 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:58.881276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b35a4-00ed-4f97-877e-481e34dda5e2 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:58.885300 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b3368-6053-4b49-9283-23c6fab631c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:58.885910 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b3368-6053-4b49-9283-23c6fab631c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:58.886186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b3368-6053-4b49-9283-23c6fab631c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:58.886456 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b3368-6053-4b49-9283-23c6fab631c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:58.895965 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b3368-6053-4b49-9283-23c6fab631c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:58.897428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b3368-6053-4b49-9283-23c6fab631c0 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:58.906656 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1356/2677] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:09:58 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:09:59.067940 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f99268-7e73-4a88-b098-77a35ff1f104 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:09:59.077403 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f99268-7e73-4a88-b098-77a35ff1f104 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:09:59.081012 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5def54-c0a2-48bf-bc0e-5a0faa71c269 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:09:59.081616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5def54-c0a2-48bf-bc0e-5a0faa71c269 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:09:59.081919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5def54-c0a2-48bf-bc0e-5a0faa71c269 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:09:59.082193 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5def54-c0a2-48bf-bc0e-5a0faa71c269 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:09:59.098384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:09:59.137588 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5def54-c0a2-48bf-bc0e-5a0faa71c269 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:09:59.139021 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5def54-c0a2-48bf-bc0e-5a0faa71c269 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:09:59.147716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1322/2678] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:09:59 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:10:05.215121 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b3368-6053-4b49-9283-23c6fab631c0 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:05.216520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-aa3b3368-6053-4b49-9283-23c6fab631c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:05.292762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b3368-6053-4b49-9283-23c6fab631c0 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:05.297760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544f3c8-57c5-4973-b83d-1c33ad021bc9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2244655ba304b9c980b5e68772eefc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:05.298361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544f3c8-57c5-4973-b83d-1c33ad021bc9 service placement] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:05.298638 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544f3c8-57c5-4973-b83d-1c33ad021bc9 service placement] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:05.298914 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544f3c8-57c5-4973-b83d-1c33ad021bc9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:05.300672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0544f3c8-57c5-4973-b83d-1c33ad021bc9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:05.311649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544f3c8-57c5-4973-b83d-1c33ad021bc9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:05.313718 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544f3c8-57c5-4973-b83d-1c33ad021bc9 service placement] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:05.323512 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1357/2679] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:10:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:10:06.230793 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5def54-c0a2-48bf-bc0e-5a0faa71c269 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:06.291358 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5def54-c0a2-48bf-bc0e-5a0faa71c269 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:06.295754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28dd5e-5c49-4389-8689-2048a402102b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:06.297014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28dd5e-5c49-4389-8689-2048a402102b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:06.297292 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28dd5e-5c49-4389-8689-2048a402102b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:06.297564 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28dd5e-5c49-4389-8689-2048a402102b admin admin] PATH_INFO: `/v3/users/3ffa2a2bfcbd4e819139418c0f35fb75` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:06.298700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28dd5e-5c49-4389-8689-2048a402102b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ffa2a2bfcbd4e819139418c0f35fb75)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:06.300636 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28dd5e-5c49-4389-8689-2048a402102b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:06.345891 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-cc28dd5e-5c49-4389-8689-2048a402102b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ffa2a2bfcbd4e819139418c0f35fb75'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:06.352848 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-cc28dd5e-5c49-4389-8689-2048a402102b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ffa2a2bfcbd4e819139418c0f35fb75'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:06.365106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-cc28dd5e-5c49-4389-8689-2048a402102b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ffa2a2bfcbd4e819139418c0f35fb75'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:06.365756 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-cc28dd5e-5c49-4389-8689-2048a402102b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ffa2a2bfcbd4e819139418c0f35fb75'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:06.366345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-cc28dd5e-5c49-4389-8689-2048a402102b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ffa2a2bfcbd4e819139418c0f35fb75'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:06.378846 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1323/2680] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:10:06 2022] DELETE /identity/v3/users/3ffa2a2bfcbd4e819139418c0f35fb75 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:06.383597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b23c11-c75b-4f1e-8d01-3727790eeff2 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:06.384101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b23c11-c75b-4f1e-8d01-3727790eeff2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:06.384194 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b23c11-c75b-4f1e-8d01-3727790eeff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:06.402607 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e8b23c11-c75b-4f1e-8d01-3727790eeff2 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:06.403896 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e8b23c11-c75b-4f1e-8d01-3727790eeff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:06.465260 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1358/2681] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:06.475977 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28dd5e-5c49-4389-8689-2048a402102b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:06.477379 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-cc28dd5e-5c49-4389-8689-2048a402102b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:06.517981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:06.535625 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28dd5e-5c49-4389-8689-2048a402102b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:06.539284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed273627-6e35-4671-8663-57c2ae87b328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:06.539888 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed273627-6e35-4671-8663-57c2ae87b328 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:06.540166 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed273627-6e35-4671-8663-57c2ae87b328 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:06.540480 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed273627-6e35-4671-8663-57c2ae87b328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:06.549281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed273627-6e35-4671-8663-57c2ae87b328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:06.550852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed273627-6e35-4671-8663-57c2ae87b328 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:06.560497 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1324/2682] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:10:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:10:06.828233 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b23c11-c75b-4f1e-8d01-3727790eeff2 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:06.829823 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e8b23c11-c75b-4f1e-8d01-3727790eeff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:06.846114 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b23c11-c75b-4f1e-8d01-3727790eeff2 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:06.851419 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53559f1-d989-46b2-a370-d5d807412704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:06.852076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53559f1-d989-46b2-a370-d5d807412704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:06.852382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53559f1-d989-46b2-a370-d5d807412704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:06.853267 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53559f1-d989-46b2-a370-d5d807412704 admin admin] PATH_INFO: `/v3/projects/0e1fd4032187401696c16097246be4da` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:06.854731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53559f1-d989-46b2-a370-d5d807412704 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e1fd4032187401696c16097246be4da)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:06.856671 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53559f1-d989-46b2-a370-d5d807412704 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:06.859774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:06.905554 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c53559f1-d989-46b2-a370-d5d807412704 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e1fd4032187401696c16097246be4da', 'request_id': 'req-c53559f1-d989-46b2-a370-d5d807412704'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:06.911053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c53559f1-d989-46b2-a370-d5d807412704 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e1fd4032187401696c16097246be4da', 'request_id': 'req-c53559f1-d989-46b2-a370-d5d807412704'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:06.918293 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c53559f1-d989-46b2-a370-d5d807412704 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e1fd4032187401696c16097246be4da', 'request_id': 'req-c53559f1-d989-46b2-a370-d5d807412704'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:06.929383 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c53559f1-d989-46b2-a370-d5d807412704 admin admin] The token cache is being invalidate because project 0e1fd4032187401696c16097246be4da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:10:06.929671 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c53559f1-d989-46b2-a370-d5d807412704 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:06.930315 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c53559f1-d989-46b2-a370-d5d807412704 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:06.931803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1359/2683] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:10:06 2022] DELETE /identity/v3/projects/0e1fd4032187401696c16097246be4da => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:06.935714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed273627-6e35-4671-8663-57c2ae87b328 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:06.947609 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed273627-6e35-4671-8663-57c2ae87b328 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:07.003538 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acd5eb-2241-483d-a6cb-77869c034650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:07.004179 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acd5eb-2241-483d-a6cb-77869c034650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:07.004494 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acd5eb-2241-483d-a6cb-77869c034650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:07.004774 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acd5eb-2241-483d-a6cb-77869c034650 admin admin] PATH_INFO: `/v3/users/33752a1716fa494488c689fd2343ebb5` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:07.005934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4acd5eb-2241-483d-a6cb-77869c034650 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33752a1716fa494488c689fd2343ebb5)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:07.007886 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4acd5eb-2241-483d-a6cb-77869c034650 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:07.056575 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c4acd5eb-2241-483d-a6cb-77869c034650 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33752a1716fa494488c689fd2343ebb5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:07.063818 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c4acd5eb-2241-483d-a6cb-77869c034650 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33752a1716fa494488c689fd2343ebb5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:07.075609 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c4acd5eb-2241-483d-a6cb-77869c034650 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33752a1716fa494488c689fd2343ebb5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:07.076362 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c4acd5eb-2241-483d-a6cb-77869c034650 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33752a1716fa494488c689fd2343ebb5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:07.076991 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c4acd5eb-2241-483d-a6cb-77869c034650 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33752a1716fa494488c689fd2343ebb5'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:07.089948 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1325/2684] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:10:06 2022] DELETE /identity/v3/users/33752a1716fa494488c689fd2343ebb5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:07.373085 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53559f1-d989-46b2-a370-d5d807412704 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:07.375220 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c53559f1-d989-46b2-a370-d5d807412704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:07.388419 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53559f1-d989-46b2-a370-d5d807412704 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:07.444370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4326fe4d-70df-4528-88d4-5373d5c3db70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:07.444928 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326fe4d-70df-4528-88d4-5373d5c3db70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:07.445209 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326fe4d-70df-4528-88d4-5373d5c3db70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:07.445484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326fe4d-70df-4528-88d4-5373d5c3db70 admin admin] PATH_INFO: `/v3/projects/8b82b1a86e2c494588b140bd68e8be50` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:07.446629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4326fe4d-70df-4528-88d4-5373d5c3db70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b82b1a86e2c494588b140bd68e8be50)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:07.448565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4326fe4d-70df-4528-88d4-5373d5c3db70 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:07.493134 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4326fe4d-70df-4528-88d4-5373d5c3db70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b82b1a86e2c494588b140bd68e8be50', 'request_id': 'req-4326fe4d-70df-4528-88d4-5373d5c3db70'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:07.498560 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4326fe4d-70df-4528-88d4-5373d5c3db70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b82b1a86e2c494588b140bd68e8be50', 'request_id': 'req-4326fe4d-70df-4528-88d4-5373d5c3db70'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:07.506672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4326fe4d-70df-4528-88d4-5373d5c3db70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b82b1a86e2c494588b140bd68e8be50', 'request_id': 'req-4326fe4d-70df-4528-88d4-5373d5c3db70'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:07.518025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4326fe4d-70df-4528-88d4-5373d5c3db70 admin admin] The token cache is being invalidate because project 8b82b1a86e2c494588b140bd68e8be50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:10:07.518311 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4326fe4d-70df-4528-88d4-5373d5c3db70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:07.518949 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4326fe4d-70df-4528-88d4-5373d5c3db70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:07.520488 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1360/2685] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:10:07 2022] DELETE /identity/v3/projects/8b82b1a86e2c494588b140bd68e8be50 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:07.525359 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8465da-89a8-44ae-9bae-94b0be00cafb None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:07.525815 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8465da-89a8-44ae-9bae-94b0be00cafb None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:07.525903 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8465da-89a8-44ae-9bae-94b0be00cafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:07.545454 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-6b8465da-89a8-44ae-9bae-94b0be00cafb None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:07.547383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6b8465da-89a8-44ae-9bae-94b0be00cafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:07.598084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:07.609817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1326/2686] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 13:10:07.617611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad2e90-f4f4-4460-92ce-5107e252024c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:07.617934 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad2e90-f4f4-4460-92ce-5107e252024c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:07.618228 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad2e90-f4f4-4460-92ce-5107e252024c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:07.637533 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-78ad2e90-f4f4-4460-92ce-5107e252024c None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:07.638757 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-78ad2e90-f4f4-4460-92ce-5107e252024c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:07.647189 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1361/2687] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:07.651445 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8465da-89a8-44ae-9bae-94b0be00cafb None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:07.662174 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8465da-89a8-44ae-9bae-94b0be00cafb None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:07.666795 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624c43b-6484-49f5-a776-902e0e3182ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:07.667398 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624c43b-6484-49f5-a776-902e0e3182ba admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:07.667679 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624c43b-6484-49f5-a776-902e0e3182ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:07.667950 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624c43b-6484-49f5-a776-902e0e3182ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:07.668694 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624c43b-6484-49f5-a776-902e0e3182ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:10:07.668998 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624c43b-6484-49f5-a776-902e0e3182ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:07.670596 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624c43b-6484-49f5-a776-902e0e3182ba admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:07.686092 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1327/2688] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:10:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 13:10:07.687151 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad2e90-f4f4-4460-92ce-5107e252024c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:07.696662 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad2e90-f4f4-4460-92ce-5107e252024c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:07.701254 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145e484-6d5e-45e7-87bf-d1570fb7f5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:07.701860 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145e484-6d5e-45e7-87bf-d1570fb7f5ec admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:07.702139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145e484-6d5e-45e7-87bf-d1570fb7f5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:07.702413 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145e484-6d5e-45e7-87bf-d1570fb7f5ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:07.703087 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2145e484-6d5e-45e7-87bf-d1570fb7f5ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:07.705013 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2145e484-6d5e-45e7-87bf-d1570fb7f5ec admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:07.720366 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:07.735142 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1362/2689] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:10:07 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:10:07.739373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624c43b-6484-49f5-a776-902e0e3182ba admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:07.749636 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624c43b-6484-49f5-a776-902e0e3182ba admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:07.804069 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57748375-4ee4-4800-bac2-8ae154488183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:07.804686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57748375-4ee4-4800-bac2-8ae154488183 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:07.804962 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57748375-4ee4-4800-bac2-8ae154488183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:07.805233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57748375-4ee4-4800-bac2-8ae154488183 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:07.805891 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57748375-4ee4-4800-bac2-8ae154488183 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:07.807776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57748375-4ee4-4800-bac2-8ae154488183 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:07.827437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1328/2690] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:10:07 2022] POST /identity/v3/users => generated 287 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:10:07.831629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145e484-6d5e-45e7-87bf-d1570fb7f5ec admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:07.841971 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145e484-6d5e-45e7-87bf-d1570fb7f5ec admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:07.847198 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ff765-56f6-4927-b14d-924b61ab3ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:07.847796 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ff765-56f6-4927-b14d-924b61ab3ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:07.848072 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ff765-56f6-4927-b14d-924b61ab3ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:07.848368 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ff765-56f6-4927-b14d-924b61ab3ea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:07.849321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ff765-56f6-4927-b14d-924b61ab3ea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:07.850986 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ff765-56f6-4927-b14d-924b61ab3ea2 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:07.861383 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1363/2691] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:10:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 13:10:07.865138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57748375-4ee4-4800-bac2-8ae154488183 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:07.874812 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57748375-4ee4-4800-bac2-8ae154488183 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:07.879360 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f269d-ccab-4bb3-9754-c30b46e8f070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:07.879955 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f269d-ccab-4bb3-9754-c30b46e8f070 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:07.880231 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f269d-ccab-4bb3-9754-c30b46e8f070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:07.880534 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f269d-ccab-4bb3-9754-c30b46e8f070 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:07.881220 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f269d-ccab-4bb3-9754-c30b46e8f070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:07.882782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f269d-ccab-4bb3-9754-c30b46e8f070 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:07.892887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1329/2692] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:10:07 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:10:07.897137 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ff765-56f6-4927-b14d-924b61ab3ea2 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:07.907324 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ff765-56f6-4927-b14d-924b61ab3ea2 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:07.911880 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df20945-2862-432b-9b0a-e267230bef95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:07.912565 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df20945-2862-432b-9b0a-e267230bef95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:07.912845 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df20945-2862-432b-9b0a-e267230bef95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:07.913142 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df20945-2862-432b-9b0a-e267230bef95 admin admin] PATH_INFO: `/v3/projects/8f8ad8bc9879474b8de3383986e5bff0/users/3bc8a3a20d1044ad8e5e71b49740c2d3/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:07.928195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df20945-2862-432b-9b0a-e267230bef95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f8ad8bc9879474b8de3383986e5bff0, user_id=3bc8a3a20d1044ad8e5e71b49740c2d3, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:07.930857 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df20945-2862-432b-9b0a-e267230bef95 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:07.942737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1364/2693] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:10:07 2022] PUT /identity/v3/projects/8f8ad8bc9879474b8de3383986e5bff0/users/3bc8a3a20d1044ad8e5e71b49740c2d3/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:07.947702 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa69bc-d198-407b-a464-7c1c49d0ab6b None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:07.947927 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa69bc-d198-407b-a464-7c1c49d0ab6b None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:07.948195 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa69bc-d198-407b-a464-7c1c49d0ab6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:07.967214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-aafa69bc-d198-407b-a464-7c1c49d0ab6b None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:07.969017 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-aafa69bc-d198-407b-a464-7c1c49d0ab6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:08.026170 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1330/2694] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:08.036206 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df20945-2862-432b-9b0a-e267230bef95 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:08.037859 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8df20945-2862-432b-9b0a-e267230bef95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:08.095841 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df20945-2862-432b-9b0a-e267230bef95 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:08.099697 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0477828-42ac-4cd5-9f8f-29aee5aebb7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:08.100327 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0477828-42ac-4cd5-9f8f-29aee5aebb7e service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:08.100612 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0477828-42ac-4cd5-9f8f-29aee5aebb7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:08.100885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0477828-42ac-4cd5-9f8f-29aee5aebb7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:08.110004 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0477828-42ac-4cd5-9f8f-29aee5aebb7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:08.111451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0477828-42ac-4cd5-9f8f-29aee5aebb7e service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:08.120926 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1365/2695] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:10:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:10:08.527061 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa69bc-d198-407b-a464-7c1c49d0ab6b None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:08.528604 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-aafa69bc-d198-407b-a464-7c1c49d0ab6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:08.541027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa69bc-d198-407b-a464-7c1c49d0ab6b None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:08.545708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067205d9-3869-497b-8f88-e5953da3b2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:08.546354 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067205d9-3869-497b-8f88-e5953da3b2e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:08.546634 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067205d9-3869-497b-8f88-e5953da3b2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:08.546910 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067205d9-3869-497b-8f88-e5953da3b2e6 admin admin] PATH_INFO: `/v3/users/b461b4df4c62496385b9a059a83a6acb` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:08.548073 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067205d9-3869-497b-8f88-e5953da3b2e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b461b4df4c62496385b9a059a83a6acb)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:08.550063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067205d9-3869-497b-8f88-e5953da3b2e6 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:08.602026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-067205d9-3869-497b-8f88-e5953da3b2e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b461b4df4c62496385b9a059a83a6acb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:08.609333 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-067205d9-3869-497b-8f88-e5953da3b2e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b461b4df4c62496385b9a059a83a6acb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:08.614353 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:08.623138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-067205d9-3869-497b-8f88-e5953da3b2e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b461b4df4c62496385b9a059a83a6acb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:08.624024 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-067205d9-3869-497b-8f88-e5953da3b2e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b461b4df4c62496385b9a059a83a6acb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:08.625496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-067205d9-3869-497b-8f88-e5953da3b2e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b461b4df4c62496385b9a059a83a6acb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:08.638070 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1331/2696] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:10:08 2022] DELETE /identity/v3/users/b461b4df4c62496385b9a059a83a6acb => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:08.643688 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b32821-9831-4aee-9eb9-e8d010e79c4e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:08.644175 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b32821-9831-4aee-9eb9-e8d010e79c4e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:08.644252 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b32821-9831-4aee-9eb9-e8d010e79c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:08.664600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-47b32821-9831-4aee-9eb9-e8d010e79c4e None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:08.665897 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-47b32821-9831-4aee-9eb9-e8d010e79c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:08.735347 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1366/2697] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:08.746561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067205d9-3869-497b-8f88-e5953da3b2e6 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:08.748098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-067205d9-3869-497b-8f88-e5953da3b2e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:08.757866 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc06d90-1268-4c4e-a917-5201b1e3835b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:08.758148 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc06d90-1268-4c4e-a917-5201b1e3835b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:08.758419 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc06d90-1268-4c4e-a917-5201b1e3835b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:08.791162 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1bc06d90-1268-4c4e-a917-5201b1e3835b None None] MFA Rules not processed for user `3bc8a3a20d1044ad8e5e71b49740c2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:08.793411 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1bc06d90-1268-4c4e-a917-5201b1e3835b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:08.820829 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067205d9-3869-497b-8f88-e5953da3b2e6 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:08.824810 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c1330-4a4a-4e56-9e78-840483ac3200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:08.825553 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c1330-4a4a-4e56-9e78-840483ac3200 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:08.825933 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c1330-4a4a-4e56-9e78-840483ac3200 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:08.826315 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c1330-4a4a-4e56-9e78-840483ac3200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:08.836867 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698c1330-4a4a-4e56-9e78-840483ac3200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:08.838449 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698c1330-4a4a-4e56-9e78-840483ac3200 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:08.850279 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1332/2698] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:10:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:10:08.864062 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:08.884624 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1367/2699] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 13:10:08.889156 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c1330-4a4a-4e56-9e78-840483ac3200 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:08.890797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-698c1330-4a4a-4e56-9e78-840483ac3200 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:08.903675 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c1330-4a4a-4e56-9e78-840483ac3200 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:08.908660 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e130ad-b1ca-49f7-85e9-419926fe3d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:08.909453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e130ad-b1ca-49f7-85e9-419926fe3d94 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:08.909670 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e130ad-b1ca-49f7-85e9-419926fe3d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:08.909782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e130ad-b1ca-49f7-85e9-419926fe3d94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:08.910441 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e130ad-b1ca-49f7-85e9-419926fe3d94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:08.912372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e130ad-b1ca-49f7-85e9-419926fe3d94 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:08.942316 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1333/2700] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:10:08 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:10:08.946032 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc06d90-1268-4c4e-a917-5201b1e3835b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:08.957012 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc06d90-1268-4c4e-a917-5201b1e3835b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:09.022860 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c9914-ef12-41fb-a812-640263d935fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:09.023378 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c9914-ef12-41fb-a812-640263d935fc admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:09.023654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c9914-ef12-41fb-a812-640263d935fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:09.023922 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c9914-ef12-41fb-a812-640263d935fc admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:09.024622 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c9914-ef12-41fb-a812-640263d935fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:09.026499 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c9914-ef12-41fb-a812-640263d935fc admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:09.047720 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1368/2701] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:10:08 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:10:09.051530 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e130ad-b1ca-49f7-85e9-419926fe3d94 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:09.062512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e130ad-b1ca-49f7-85e9-419926fe3d94 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:09.067179 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb85a9-2d7c-4bba-a474-2fc0b3899651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:09.067851 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb85a9-2d7c-4bba-a474-2fc0b3899651 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:09.068193 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb85a9-2d7c-4bba-a474-2fc0b3899651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:09.068563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb85a9-2d7c-4bba-a474-2fc0b3899651 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:09.069412 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb85a9-2d7c-4bba-a474-2fc0b3899651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:09.071051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb85a9-2d7c-4bba-a474-2fc0b3899651 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:09.082601 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1334/2702] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:10:09.086713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c9914-ef12-41fb-a812-640263d935fc admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:09.097235 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c9914-ef12-41fb-a812-640263d935fc admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:09.102211 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31743f4-4cb9-40a9-ad29-5adb43df79a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:09.102852 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31743f4-4cb9-40a9-ad29-5adb43df79a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:09.103132 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31743f4-4cb9-40a9-ad29-5adb43df79a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:09.103406 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31743f4-4cb9-40a9-ad29-5adb43df79a7 admin admin] PATH_INFO: `/v3/projects/a718128833e24d10ada84a5573181680/users/97c00b665ccc41569089e3d66d911091/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:09.118998 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31743f4-4cb9-40a9-ad29-5adb43df79a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a718128833e24d10ada84a5573181680, user_id=97c00b665ccc41569089e3d66d911091, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:09.121865 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31743f4-4cb9-40a9-ad29-5adb43df79a7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:09.138750 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1369/2703] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:10:09 2022] PUT /identity/v3/projects/a718128833e24d10ada84a5573181680/users/97c00b665ccc41569089e3d66d911091/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:10:09.142810 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb85a9-2d7c-4bba-a474-2fc0b3899651 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:09.157575 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb85a9-2d7c-4bba-a474-2fc0b3899651 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:09.175670 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31743f4-4cb9-40a9-ad29-5adb43df79a7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:09.178421 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a31743f4-4cb9-40a9-ad29-5adb43df79a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:09.222377 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e2cf9-19c4-4df2-8a0a-42b12eb608b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:09.223247 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e2cf9-19c4-4df2-8a0a-42b12eb608b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:09.223838 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e2cf9-19c4-4df2-8a0a-42b12eb608b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:09.224210 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e2cf9-19c4-4df2-8a0a-42b12eb608b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:09.225032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e2cf9-19c4-4df2-8a0a-42b12eb608b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:09.226712 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e2cf9-19c4-4df2-8a0a-42b12eb608b3 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:09.239272 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1335/2704] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:10:09.243174 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e2cf9-19c4-4df2-8a0a-42b12eb608b3 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:09.253099 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e2cf9-19c4-4df2-8a0a-42b12eb608b3 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:09.257841 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bea819-8372-4154-a630-ca9f6cc16826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:09.258543 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bea819-8372-4154-a630-ca9f6cc16826 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:09.258821 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bea819-8372-4154-a630-ca9f6cc16826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:09.259098 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bea819-8372-4154-a630-ca9f6cc16826 admin admin] PATH_INFO: `/v3/projects/a718128833e24d10ada84a5573181680/users/97c00b665ccc41569089e3d66d911091/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:09.261021 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bea819-8372-4154-a630-ca9f6cc16826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a718128833e24d10ada84a5573181680, user_id=97c00b665ccc41569089e3d66d911091, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:09.263610 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bea819-8372-4154-a630-ca9f6cc16826 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:09.267535 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31743f4-4cb9-40a9-ad29-5adb43df79a7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:09.273810 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1336/2705] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:10:09 2022] PUT /identity/v3/projects/a718128833e24d10ada84a5573181680/users/97c00b665ccc41569089e3d66d911091/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:10:09.325781 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550d1f0-8c11-4185-92c6-04785fac6950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:09.326418 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550d1f0-8c11-4185-92c6-04785fac6950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:09.326694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550d1f0-8c11-4185-92c6-04785fac6950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:09.326964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550d1f0-8c11-4185-92c6-04785fac6950 admin admin] PATH_INFO: `/v3/projects/32ddc615629d43c2956984b1a2f58a04` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:09.328154 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b550d1f0-8c11-4185-92c6-04785fac6950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32ddc615629d43c2956984b1a2f58a04)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:09.330104 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b550d1f0-8c11-4185-92c6-04785fac6950 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:09.336615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:09.380544 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b550d1f0-8c11-4185-92c6-04785fac6950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32ddc615629d43c2956984b1a2f58a04', 'request_id': 'req-b550d1f0-8c11-4185-92c6-04785fac6950'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:09.386012 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b550d1f0-8c11-4185-92c6-04785fac6950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32ddc615629d43c2956984b1a2f58a04', 'request_id': 'req-b550d1f0-8c11-4185-92c6-04785fac6950'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:09.393643 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b550d1f0-8c11-4185-92c6-04785fac6950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32ddc615629d43c2956984b1a2f58a04', 'request_id': 'req-b550d1f0-8c11-4185-92c6-04785fac6950'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:09.405830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b550d1f0-8c11-4185-92c6-04785fac6950 admin admin] The token cache is being invalidate because project 32ddc615629d43c2956984b1a2f58a04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:10:09.406084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b550d1f0-8c11-4185-92c6-04785fac6950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:09.406741 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b550d1f0-8c11-4185-92c6-04785fac6950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:09.411187 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1370/2706] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:10:09 2022] DELETE /identity/v3/projects/32ddc615629d43c2956984b1a2f58a04 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:10:09.412379 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bea819-8372-4154-a630-ca9f6cc16826 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:09.414062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-31bea819-8372-4154-a630-ca9f6cc16826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:09.428062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bea819-8372-4154-a630-ca9f6cc16826 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:09.459249 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:09.489631 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8c8e5-0786-4c07-a2c4-0dfb634cf9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:09.490270 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8c8e5-0786-4c07-a2c4-0dfb634cf9d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:09.490546 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8c8e5-0786-4c07-a2c4-0dfb634cf9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:09.490816 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8c8e5-0786-4c07-a2c4-0dfb634cf9d0 admin admin] PATH_INFO: `/v3/users/df08c90271404a17b647e4e30e705d76` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:09.491973 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8c8e5-0786-4c07-a2c4-0dfb634cf9d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df08c90271404a17b647e4e30e705d76)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:09.493912 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8c8e5-0786-4c07-a2c4-0dfb634cf9d0 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:09.543336 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d1a8c8e5-0786-4c07-a2c4-0dfb634cf9d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df08c90271404a17b647e4e30e705d76'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:09.550275 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d1a8c8e5-0786-4c07-a2c4-0dfb634cf9d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df08c90271404a17b647e4e30e705d76'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:09.562519 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d1a8c8e5-0786-4c07-a2c4-0dfb634cf9d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df08c90271404a17b647e4e30e705d76'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:09.563414 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d1a8c8e5-0786-4c07-a2c4-0dfb634cf9d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df08c90271404a17b647e4e30e705d76'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:09.564166 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-d1a8c8e5-0786-4c07-a2c4-0dfb634cf9d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df08c90271404a17b647e4e30e705d76'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:09.581902 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1337/2707] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:10:09 2022] DELETE /identity/v3/users/df08c90271404a17b647e4e30e705d76 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:10.187936 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550d1f0-8c11-4185-92c6-04785fac6950 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:10.198882 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550d1f0-8c11-4185-92c6-04785fac6950 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:10.229716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8262b5d-c788-4043-ba14-2f760f5e5541 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:10.230010 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8262b5d-c788-4043-ba14-2f760f5e5541 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:10.230816 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8262b5d-c788-4043-ba14-2f760f5e5541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:10.262870 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c8262b5d-c788-4043-ba14-2f760f5e5541 None None] MFA Rules not processed for user `97c00b665ccc41569089e3d66d911091`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:10.264198 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c8262b5d-c788-4043-ba14-2f760f5e5541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:10.270424 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb2683-a896-4c11-b9bc-ee3b68b4a279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:10.271071 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb2683-a896-4c11-b9bc-ee3b68b4a279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:10.271348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb2683-a896-4c11-b9bc-ee3b68b4a279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:10.271618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb2683-a896-4c11-b9bc-ee3b68b4a279 admin admin] PATH_INFO: `/v3/projects/a1b230e611764afeaf25698ec1efe047` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:10.272815 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb2683-a896-4c11-b9bc-ee3b68b4a279 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1b230e611764afeaf25698ec1efe047)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:10.274765 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb2683-a896-4c11-b9bc-ee3b68b4a279 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:10.334799 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-91fb2683-a896-4c11-b9bc-ee3b68b4a279 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1b230e611764afeaf25698ec1efe047', 'request_id': 'req-91fb2683-a896-4c11-b9bc-ee3b68b4a279'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:10.335496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1338/2708] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 13:10:10.341617 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-91fb2683-a896-4c11-b9bc-ee3b68b4a279 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1b230e611764afeaf25698ec1efe047', 'request_id': 'req-91fb2683-a896-4c11-b9bc-ee3b68b4a279'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:10.348212 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8262b5d-c788-4043-ba14-2f760f5e5541 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:10.350144 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c8262b5d-c788-4043-ba14-2f760f5e5541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:10.350292 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-91fb2683-a896-4c11-b9bc-ee3b68b4a279 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1b230e611764afeaf25698ec1efe047', 'request_id': 'req-91fb2683-a896-4c11-b9bc-ee3b68b4a279'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:10.365857 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-91fb2683-a896-4c11-b9bc-ee3b68b4a279 admin admin] The token cache is being invalidate because project a1b230e611764afeaf25698ec1efe047 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:10:10.366163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-91fb2683-a896-4c11-b9bc-ee3b68b4a279 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:10.366786 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-91fb2683-a896-4c11-b9bc-ee3b68b4a279 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:10.371593 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1371/2709] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:10:10 2022] DELETE /identity/v3/projects/a1b230e611764afeaf25698ec1efe047 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:10.373061 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416a847-c48a-4d8c-abf3-3c85c6527674 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:10.373339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416a847-c48a-4d8c-abf3-3c85c6527674 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:10.373611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416a847-c48a-4d8c-abf3-3c85c6527674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:10.394605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:10.396160 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f416a847-c48a-4d8c-abf3-3c85c6527674 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:10.397403 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f416a847-c48a-4d8c-abf3-3c85c6527674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:10.417038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8262b5d-c788-4043-ba14-2f760f5e5541 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:10.419341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:10.461803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1372/2710] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 13:10:10.469739 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e682d0-301b-4aab-a216-bab6165c4962 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:10.469897 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fcdcd-489f-429c-ac25-d0251d1d0ea3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:10.470041 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e682d0-301b-4aab-a216-bab6165c4962 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:10.470274 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e682d0-301b-4aab-a216-bab6165c4962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:10.470396 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fcdcd-489f-429c-ac25-d0251d1d0ea3 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:10.470680 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fcdcd-489f-429c-ac25-d0251d1d0ea3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:10.470948 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fcdcd-489f-429c-ac25-d0251d1d0ea3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:10.472717 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-325fcdcd-489f-429c-ac25-d0251d1d0ea3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:10.489827 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-21e682d0-301b-4aab-a216-bab6165c4962 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:10.490992 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-21e682d0-301b-4aab-a216-bab6165c4962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:10.501217 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1373/2711] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 13:10:10.504831 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e682d0-301b-4aab-a216-bab6165c4962 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:10.513632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e682d0-301b-4aab-a216-bab6165c4962 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:10.518340 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c742e14-050f-468c-9a6c-1eeceaba6142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:10.518921 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c742e14-050f-468c-9a6c-1eeceaba6142 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:10.519195 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c742e14-050f-468c-9a6c-1eeceaba6142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:10.519463 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c742e14-050f-468c-9a6c-1eeceaba6142 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:10.520134 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c742e14-050f-468c-9a6c-1eeceaba6142 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:10:10.520471 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c742e14-050f-468c-9a6c-1eeceaba6142 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:10.522031 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c742e14-050f-468c-9a6c-1eeceaba6142 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:10.533166 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1374/2712] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:10:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 13:10:10.537446 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c742e14-050f-468c-9a6c-1eeceaba6142 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:10.537631 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325fcdcd-489f-429c-ac25-d0251d1d0ea3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:10.539072 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325fcdcd-489f-429c-ac25-d0251d1d0ea3 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:10.547180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c742e14-050f-468c-9a6c-1eeceaba6142 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:10.548793 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1339/2713] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:10:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 13:10:10.551921 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1858ce-4f2a-4c8e-ac39-6213ba571820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:10.552530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1858ce-4f2a-4c8e-ac39-6213ba571820 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:10.552806 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1858ce-4f2a-4c8e-ac39-6213ba571820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:10.553078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1858ce-4f2a-4c8e-ac39-6213ba571820 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:10.553723 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1858ce-4f2a-4c8e-ac39-6213ba571820 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:10.555573 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1858ce-4f2a-4c8e-ac39-6213ba571820 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:10.587818 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1375/2714] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:10:10 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:10:10.589404 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fcdcd-489f-429c-ac25-d0251d1d0ea3 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:10.601990 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fcdcd-489f-429c-ac25-d0251d1d0ea3 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:10.659097 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f141e3-fcdb-4ea5-aa15-73c9b9e024df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:10.659722 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f141e3-fcdb-4ea5-aa15-73c9b9e024df admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:10.659999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f141e3-fcdb-4ea5-aa15-73c9b9e024df admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:10.660274 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f141e3-fcdb-4ea5-aa15-73c9b9e024df admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:10.660959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f141e3-fcdb-4ea5-aa15-73c9b9e024df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:10.662883 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f141e3-fcdb-4ea5-aa15-73c9b9e024df admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:10.685502 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1340/2715] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:10:10 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:10:10.690783 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1858ce-4f2a-4c8e-ac39-6213ba571820 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:10.703274 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1858ce-4f2a-4c8e-ac39-6213ba571820 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:10.708372 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd5df4-73f5-4cda-88a4-97564a702348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:10.708979 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd5df4-73f5-4cda-88a4-97564a702348 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:10.709311 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd5df4-73f5-4cda-88a4-97564a702348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:10.709520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd5df4-73f5-4cda-88a4-97564a702348 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:10.710249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd5df4-73f5-4cda-88a4-97564a702348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:10.711818 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd5df4-73f5-4cda-88a4-97564a702348 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:10.722601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1376/2716] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 13:10:10.729826 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f141e3-fcdb-4ea5-aa15-73c9b9e024df admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:10.742691 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f141e3-fcdb-4ea5-aa15-73c9b9e024df admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:10.747670 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d099f-d1f4-4e70-8137-dfac3fa85d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:10.748274 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d099f-d1f4-4e70-8137-dfac3fa85d71 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:10.748587 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d099f-d1f4-4e70-8137-dfac3fa85d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:10.748863 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d099f-d1f4-4e70-8137-dfac3fa85d71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:10.749652 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183d099f-d1f4-4e70-8137-dfac3fa85d71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:10.751476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183d099f-d1f4-4e70-8137-dfac3fa85d71 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:10.762389 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1341/2717] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:10:10.766749 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd5df4-73f5-4cda-88a4-97564a702348 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:10.777495 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd5df4-73f5-4cda-88a4-97564a702348 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:10.782879 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70364aa-4f77-419a-a957-60cc2330af69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:10.783549 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70364aa-4f77-419a-a957-60cc2330af69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:10.783828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70364aa-4f77-419a-a957-60cc2330af69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:10.784087 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70364aa-4f77-419a-a957-60cc2330af69 admin admin] PATH_INFO: `/v3/projects/54bb9fae3a4146589a3e4e4e043f1ebc/users/c03109216db649b98ea1a21d4d5b4133/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:10.803248 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70364aa-4f77-419a-a957-60cc2330af69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54bb9fae3a4146589a3e4e4e043f1ebc, user_id=c03109216db649b98ea1a21d4d5b4133, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:10.805927 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70364aa-4f77-419a-a957-60cc2330af69 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:10.819770 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1377/2718] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:10:10 2022] PUT /identity/v3/projects/54bb9fae3a4146589a3e4e4e043f1ebc/users/c03109216db649b98ea1a21d4d5b4133/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:10:10.831313 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f297008-64a5-44b6-a18f-7695ff19f959 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:10.831567 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f297008-64a5-44b6-a18f-7695ff19f959 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:10.831849 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f297008-64a5-44b6-a18f-7695ff19f959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:10.863317 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-0f297008-64a5-44b6-a18f-7695ff19f959 None None] MFA Rules not processed for user `c03109216db649b98ea1a21d4d5b4133`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:10.864580 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0f297008-64a5-44b6-a18f-7695ff19f959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:10.877358 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:10.938512 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1342/2719] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:10.942868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70364aa-4f77-419a-a957-60cc2330af69 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:11.011539 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70364aa-4f77-419a-a957-60cc2330af69 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:11.016646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310ef7f-9fda-43e3-a098-6087ffe89ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:11.017251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310ef7f-9fda-43e3-a098-6087ffe89ff6 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:11.017527 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310ef7f-9fda-43e3-a098-6087ffe89ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:11.017799 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310ef7f-9fda-43e3-a098-6087ffe89ff6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:11.018452 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4310ef7f-9fda-43e3-a098-6087ffe89ff6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:11.020459 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4310ef7f-9fda-43e3-a098-6087ffe89ff6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:11.047090 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1378/2720] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:10:10 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:10:11.051058 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f297008-64a5-44b6-a18f-7695ff19f959 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:11.061795 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f297008-64a5-44b6-a18f-7695ff19f959 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:11.115517 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a04f7-1581-4428-8023-c06759965307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:11.116112 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a04f7-1581-4428-8023-c06759965307 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:11.116413 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a04f7-1581-4428-8023-c06759965307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:11.116683 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a04f7-1581-4428-8023-c06759965307 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:11.117347 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a04f7-1581-4428-8023-c06759965307 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:11.119212 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a04f7-1581-4428-8023-c06759965307 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:11.140186 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1343/2721] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:10:11 2022] POST /identity/v3/users => generated 300 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:10:11.145758 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310ef7f-9fda-43e3-a098-6087ffe89ff6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:11.157724 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310ef7f-9fda-43e3-a098-6087ffe89ff6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:11.162320 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457c659-eae9-42bf-88ff-97e1308052b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:11.162924 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457c659-eae9-42bf-88ff-97e1308052b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:11.163204 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457c659-eae9-42bf-88ff-97e1308052b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:11.163476 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457c659-eae9-42bf-88ff-97e1308052b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:11.164162 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3457c659-eae9-42bf-88ff-97e1308052b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:11.165774 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3457c659-eae9-42bf-88ff-97e1308052b4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:11.175978 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1379/2722] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:10:11.179978 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a04f7-1581-4428-8023-c06759965307 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:11.190010 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a04f7-1581-4428-8023-c06759965307 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:11.194505 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe66fd-c80f-47e3-ae61-c1276d84eaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:11.195153 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe66fd-c80f-47e3-ae61-c1276d84eaef admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:11.195433 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe66fd-c80f-47e3-ae61-c1276d84eaef admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:11.195703 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe66fd-c80f-47e3-ae61-c1276d84eaef admin admin] PATH_INFO: `/v3/projects/9c99fe7a12ce4989aaeae8cfa848f65e/users/2e406f3031a1458ca7f49ddced6b4bcb/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:11.210476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe66fd-c80f-47e3-ae61-c1276d84eaef admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c99fe7a12ce4989aaeae8cfa848f65e, user_id=2e406f3031a1458ca7f49ddced6b4bcb, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:11.213302 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe66fd-c80f-47e3-ae61-c1276d84eaef admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:11.227695 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1344/2723] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:10:11 2022] PUT /identity/v3/projects/9c99fe7a12ce4989aaeae8cfa848f65e/users/2e406f3031a1458ca7f49ddced6b4bcb/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:11.231781 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457c659-eae9-42bf-88ff-97e1308052b4 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:11.241800 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457c659-eae9-42bf-88ff-97e1308052b4 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:11.259726 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:11.299174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff797c-e6fb-454f-afa1-8e7c8ac4706e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:11.299777 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff797c-e6fb-454f-afa1-8e7c8ac4706e admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:11.300058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff797c-e6fb-454f-afa1-8e7c8ac4706e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:11.300356 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff797c-e6fb-454f-afa1-8e7c8ac4706e admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:11.301042 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff797c-e6fb-454f-afa1-8e7c8ac4706e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:11.302617 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff797c-e6fb-454f-afa1-8e7c8ac4706e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:11.312640 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1380/2724] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:10:11.316749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe66fd-c80f-47e3-ae61-c1276d84eaef admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:11.326787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe66fd-c80f-47e3-ae61-c1276d84eaef admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:11.331841 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48866dd6-3d65-4b6d-8a1b-0e3780c067bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:11.332505 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48866dd6-3d65-4b6d-8a1b-0e3780c067bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:11.332779 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48866dd6-3d65-4b6d-8a1b-0e3780c067bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:11.333045 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48866dd6-3d65-4b6d-8a1b-0e3780c067bf admin admin] PATH_INFO: `/v3/projects/9c99fe7a12ce4989aaeae8cfa848f65e/users/2e406f3031a1458ca7f49ddced6b4bcb/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:11.334954 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48866dd6-3d65-4b6d-8a1b-0e3780c067bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c99fe7a12ce4989aaeae8cfa848f65e, user_id=2e406f3031a1458ca7f49ddced6b4bcb, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:11.337591 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48866dd6-3d65-4b6d-8a1b-0e3780c067bf admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:11.348155 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1345/2725] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:10:11 2022] PUT /identity/v3/projects/9c99fe7a12ce4989aaeae8cfa848f65e/users/2e406f3031a1458ca7f49ddced6b4bcb/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:11.359218 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291bb85-28a6-46d3-93e7-440e4a3c2588 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:11.359501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291bb85-28a6-46d3-93e7-440e4a3c2588 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:11.359769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291bb85-28a6-46d3-93e7-440e4a3c2588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:11.386386 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2291bb85-28a6-46d3-93e7-440e4a3c2588 None None] MFA Rules not processed for user `2e406f3031a1458ca7f49ddced6b4bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:11.387563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2291bb85-28a6-46d3-93e7-440e4a3c2588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:11.452665 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1381/2726] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:11 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:11.462930 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48866dd6-3d65-4b6d-8a1b-0e3780c067bf admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:11.464663 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-48866dd6-3d65-4b6d-8a1b-0e3780c067bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:11.520195 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48866dd6-3d65-4b6d-8a1b-0e3780c067bf admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:11.523753 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d27f4-625a-4942-8455-1ec148579231 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:11.524374 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d27f4-625a-4942-8455-1ec148579231 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:11.524651 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d27f4-625a-4942-8455-1ec148579231 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:11.524918 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d27f4-625a-4942-8455-1ec148579231 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:11.534074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d27f4-625a-4942-8455-1ec148579231 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:11.535474 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d27f4-625a-4942-8455-1ec148579231 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:11.544545 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1346/2727] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:10:11 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:10:11.912368 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291bb85-28a6-46d3-93e7-440e4a3c2588 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:11.914305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2291bb85-28a6-46d3-93e7-440e4a3c2588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:11.928465 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291bb85-28a6-46d3-93e7-440e4a3c2588 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:11.932626 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc9a34-d7e5-4b9e-a63c-aa6c58450be0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:11.933237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc9a34-d7e5-4b9e-a63c-aa6c58450be0 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:11.933516 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc9a34-d7e5-4b9e-a63c-aa6c58450be0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:11.933785 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc9a34-d7e5-4b9e-a63c-aa6c58450be0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:11.935499 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e9dc9a34-d7e5-4b9e-a63c-aa6c58450be0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:12.003475 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc9a34-d7e5-4b9e-a63c-aa6c58450be0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:12.004957 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc9a34-d7e5-4b9e-a63c-aa6c58450be0 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:12.015968 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1382/2728] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:10:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:10:12.109749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d27f4-625a-4942-8455-1ec148579231 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:12.119824 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:12.174971 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d27f4-625a-4942-8455-1ec148579231 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:12.179694 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47419f43-1b05-4770-91e3-750aedb1483c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:12.180348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47419f43-1b05-4770-91e3-750aedb1483c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:12.180626 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47419f43-1b05-4770-91e3-750aedb1483c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:12.180917 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47419f43-1b05-4770-91e3-750aedb1483c admin admin] PATH_INFO: `/v3/users/c03109216db649b98ea1a21d4d5b4133` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:12.182061 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47419f43-1b05-4770-91e3-750aedb1483c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c03109216db649b98ea1a21d4d5b4133)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:12.184011 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47419f43-1b05-4770-91e3-750aedb1483c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:12.234616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-47419f43-1b05-4770-91e3-750aedb1483c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c03109216db649b98ea1a21d4d5b4133'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:12.241375 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-47419f43-1b05-4770-91e3-750aedb1483c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c03109216db649b98ea1a21d4d5b4133'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:12.253159 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-47419f43-1b05-4770-91e3-750aedb1483c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c03109216db649b98ea1a21d4d5b4133'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:12.257979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-47419f43-1b05-4770-91e3-750aedb1483c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c03109216db649b98ea1a21d4d5b4133'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:12.258578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-47419f43-1b05-4770-91e3-750aedb1483c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c03109216db649b98ea1a21d4d5b4133'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:12.270767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1347/2729] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:10:12 2022] DELETE /identity/v3/users/c03109216db649b98ea1a21d4d5b4133 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:12.275419 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389da5db-5f75-4c07-b4ba-038981a3de0b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:12.275886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389da5db-5f75-4c07-b4ba-038981a3de0b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:12.275977 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389da5db-5f75-4c07-b4ba-038981a3de0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:12.294911 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-389da5db-5f75-4c07-b4ba-038981a3de0b None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:12.296869 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-389da5db-5f75-4c07-b4ba-038981a3de0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:12.327428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:12.354264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1383/2730] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:12.364552 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47419f43-1b05-4770-91e3-750aedb1483c admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:12.365986 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-47419f43-1b05-4770-91e3-750aedb1483c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:12.422587 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47419f43-1b05-4770-91e3-750aedb1483c admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:12.426225 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6b929-1194-4a8d-9fa3-8f632b9d5e5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:12.426883 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6b929-1194-4a8d-9fa3-8f632b9d5e5f service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:12.427171 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6b929-1194-4a8d-9fa3-8f632b9d5e5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:12.427472 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6b929-1194-4a8d-9fa3-8f632b9d5e5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:12.436728 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c6b929-1194-4a8d-9fa3-8f632b9d5e5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:12.438176 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c6b929-1194-4a8d-9fa3-8f632b9d5e5f service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:12.447497 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1348/2731] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:10:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:10:12.713868 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389da5db-5f75-4c07-b4ba-038981a3de0b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:12.715257 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-389da5db-5f75-4c07-b4ba-038981a3de0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:12.728172 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389da5db-5f75-4c07-b4ba-038981a3de0b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:12.732702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e74cf3-d765-4a24-b17d-4682fdeb9c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:12.733365 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e74cf3-d765-4a24-b17d-4682fdeb9c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:12.733641 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e74cf3-d765-4a24-b17d-4682fdeb9c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:12.733907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e74cf3-d765-4a24-b17d-4682fdeb9c9c admin admin] PATH_INFO: `/v3/projects/54bb9fae3a4146589a3e4e4e043f1ebc` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:12.735041 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e74cf3-d765-4a24-b17d-4682fdeb9c9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=54bb9fae3a4146589a3e4e4e043f1ebc)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:12.736995 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e74cf3-d765-4a24-b17d-4682fdeb9c9c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:12.783947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-13e74cf3-d765-4a24-b17d-4682fdeb9c9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54bb9fae3a4146589a3e4e4e043f1ebc', 'request_id': 'req-13e74cf3-d765-4a24-b17d-4682fdeb9c9c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:12.789362 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-13e74cf3-d765-4a24-b17d-4682fdeb9c9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54bb9fae3a4146589a3e4e4e043f1ebc', 'request_id': 'req-13e74cf3-d765-4a24-b17d-4682fdeb9c9c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:12.796701 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-13e74cf3-d765-4a24-b17d-4682fdeb9c9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54bb9fae3a4146589a3e4e4e043f1ebc', 'request_id': 'req-13e74cf3-d765-4a24-b17d-4682fdeb9c9c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:12.807480 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-13e74cf3-d765-4a24-b17d-4682fdeb9c9c admin admin] The token cache is being invalidate because project 54bb9fae3a4146589a3e4e4e043f1ebc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:10:12.807742 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-13e74cf3-d765-4a24-b17d-4682fdeb9c9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:12.808395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-13e74cf3-d765-4a24-b17d-4682fdeb9c9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:12.809884 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1384/2732] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:10:12 2022] DELETE /identity/v3/projects/54bb9fae3a4146589a3e4e4e043f1ebc => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:12.814348 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6b929-1194-4a8d-9fa3-8f632b9d5e5f service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:12.823863 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6b929-1194-4a8d-9fa3-8f632b9d5e5f service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:12.852094 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:12.881036 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b80a38-5ba6-44a9-a8a8-597f73454abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:12.881668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b80a38-5ba6-44a9-a8a8-597f73454abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:12.881951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b80a38-5ba6-44a9-a8a8-597f73454abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:12.882229 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b80a38-5ba6-44a9-a8a8-597f73454abf admin admin] PATH_INFO: `/v3/users/2e406f3031a1458ca7f49ddced6b4bcb` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:12.883377 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b80a38-5ba6-44a9-a8a8-597f73454abf admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e406f3031a1458ca7f49ddced6b4bcb)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:12.885347 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b80a38-5ba6-44a9-a8a8-597f73454abf admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:12.929643 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-37b80a38-5ba6-44a9-a8a8-597f73454abf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e406f3031a1458ca7f49ddced6b4bcb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:12.937009 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-37b80a38-5ba6-44a9-a8a8-597f73454abf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e406f3031a1458ca7f49ddced6b4bcb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:12.948669 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-37b80a38-5ba6-44a9-a8a8-597f73454abf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e406f3031a1458ca7f49ddced6b4bcb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:12.949351 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-37b80a38-5ba6-44a9-a8a8-597f73454abf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e406f3031a1458ca7f49ddced6b4bcb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:12.949960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-37b80a38-5ba6-44a9-a8a8-597f73454abf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e406f3031a1458ca7f49ddced6b4bcb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:12.962149 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1349/2733] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:10:12 2022] DELETE /identity/v3/users/2e406f3031a1458ca7f49ddced6b4bcb => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:13.231052 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e74cf3-d765-4a24-b17d-4682fdeb9c9c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:13.232668 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-13e74cf3-d765-4a24-b17d-4682fdeb9c9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:13.248102 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e74cf3-d765-4a24-b17d-4682fdeb9c9c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:13.313996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474807ba-fe00-49dc-a88d-d9aa9e096959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:13.314640 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474807ba-fe00-49dc-a88d-d9aa9e096959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:13.314920 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474807ba-fe00-49dc-a88d-d9aa9e096959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:13.315237 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474807ba-fe00-49dc-a88d-d9aa9e096959 admin admin] PATH_INFO: `/v3/projects/9c99fe7a12ce4989aaeae8cfa848f65e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:13.316546 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474807ba-fe00-49dc-a88d-d9aa9e096959 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c99fe7a12ce4989aaeae8cfa848f65e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:13.319244 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474807ba-fe00-49dc-a88d-d9aa9e096959 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:13.371280 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-474807ba-fe00-49dc-a88d-d9aa9e096959 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c99fe7a12ce4989aaeae8cfa848f65e', 'request_id': 'req-474807ba-fe00-49dc-a88d-d9aa9e096959'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:13.377911 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-474807ba-fe00-49dc-a88d-d9aa9e096959 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c99fe7a12ce4989aaeae8cfa848f65e', 'request_id': 'req-474807ba-fe00-49dc-a88d-d9aa9e096959'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:13.381475 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:13.385448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-474807ba-fe00-49dc-a88d-d9aa9e096959 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c99fe7a12ce4989aaeae8cfa848f65e', 'request_id': 'req-474807ba-fe00-49dc-a88d-d9aa9e096959'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:13.402421 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-474807ba-fe00-49dc-a88d-d9aa9e096959 admin admin] The token cache is being invalidate because project 9c99fe7a12ce4989aaeae8cfa848f65e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:10:13.402703 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-474807ba-fe00-49dc-a88d-d9aa9e096959 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:13.403342 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-474807ba-fe00-49dc-a88d-d9aa9e096959 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:13.405287 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1385/2734] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:10:13 2022] DELETE /identity/v3/projects/9c99fe7a12ce4989aaeae8cfa848f65e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:13.410168 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014583d4-4fc4-424d-aa34-52544abe711d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:13.410648 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014583d4-4fc4-424d-aa34-52544abe711d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:13.410934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014583d4-4fc4-424d-aa34-52544abe711d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:13.431808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-014583d4-4fc4-424d-aa34-52544abe711d None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:13.433052 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-014583d4-4fc4-424d-aa34-52544abe711d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:13.520452 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1350/2735] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 13:10:13.523940 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39630447-3276-4460-9c0e-5cb5a1329d5b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:13.524129 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39630447-3276-4460-9c0e-5cb5a1329d5b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:13.524432 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39630447-3276-4460-9c0e-5cb5a1329d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:13.547077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-39630447-3276-4460-9c0e-5cb5a1329d5b None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:13.548452 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-39630447-3276-4460-9c0e-5cb5a1329d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:13.556907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1386/2736] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:13.562026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014583d4-4fc4-424d-aa34-52544abe711d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:13.572319 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014583d4-4fc4-424d-aa34-52544abe711d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:13.577202 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3480574-6ab2-4274-95b6-f3154b7ecc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:13.577824 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3480574-6ab2-4274-95b6-f3154b7ecc2e admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:13.578104 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3480574-6ab2-4274-95b6-f3154b7ecc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:13.578377 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3480574-6ab2-4274-95b6-f3154b7ecc2e admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:13.579152 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3480574-6ab2-4274-95b6-f3154b7ecc2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:10:13.579456 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3480574-6ab2-4274-95b6-f3154b7ecc2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:13.581207 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3480574-6ab2-4274-95b6-f3154b7ecc2e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:13.593450 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1351/2737] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:10:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 13:10:13.597973 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39630447-3276-4460-9c0e-5cb5a1329d5b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:13.609574 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39630447-3276-4460-9c0e-5cb5a1329d5b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:13.614462 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d76d2b-199c-4bf2-85c2-db5dd961f389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:13.615076 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d76d2b-199c-4bf2-85c2-db5dd961f389 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:13.615355 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d76d2b-199c-4bf2-85c2-db5dd961f389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:13.615627 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d76d2b-199c-4bf2-85c2-db5dd961f389 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:13.616339 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d76d2b-199c-4bf2-85c2-db5dd961f389 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:13.618232 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d76d2b-199c-4bf2-85c2-db5dd961f389 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:13.645087 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1387/2738] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:10:13 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:10:13.649535 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3480574-6ab2-4274-95b6-f3154b7ecc2e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:13.660014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3480574-6ab2-4274-95b6-f3154b7ecc2e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:13.692633 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:13.715821 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd12fc4-c2c9-4670-b30a-be3e4b46e0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:13.716455 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd12fc4-c2c9-4670-b30a-be3e4b46e0e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:13.716735 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd12fc4-c2c9-4670-b30a-be3e4b46e0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:13.717007 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd12fc4-c2c9-4670-b30a-be3e4b46e0e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:13.718194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd12fc4-c2c9-4670-b30a-be3e4b46e0e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:13.720086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd12fc4-c2c9-4670-b30a-be3e4b46e0e0 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:13.741975 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1352/2739] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:10:13 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:10:13.746058 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d76d2b-199c-4bf2-85c2-db5dd961f389 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:13.756399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d76d2b-199c-4bf2-85c2-db5dd961f389 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:13.761045 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475fb35-7462-48e5-ad7e-372b50ba104a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:13.761646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475fb35-7462-48e5-ad7e-372b50ba104a admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:13.761926 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475fb35-7462-48e5-ad7e-372b50ba104a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:13.762202 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475fb35-7462-48e5-ad7e-372b50ba104a admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:13.762890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475fb35-7462-48e5-ad7e-372b50ba104a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:13.764515 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475fb35-7462-48e5-ad7e-372b50ba104a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:13.774345 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1388/2740] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:10:13.778252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd12fc4-c2c9-4670-b30a-be3e4b46e0e0 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:13.789317 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd12fc4-c2c9-4670-b30a-be3e4b46e0e0 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:13.793871 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17a5b0-418c-4d6e-bf9b-51f3e771d9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:13.794467 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17a5b0-418c-4d6e-bf9b-51f3e771d9d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:13.794748 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17a5b0-418c-4d6e-bf9b-51f3e771d9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:13.795022 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17a5b0-418c-4d6e-bf9b-51f3e771d9d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:13.796365 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17a5b0-418c-4d6e-bf9b-51f3e771d9d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:13.797971 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17a5b0-418c-4d6e-bf9b-51f3e771d9d9 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:13.808250 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1353/2741] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:10:13.812365 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475fb35-7462-48e5-ad7e-372b50ba104a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:13.821962 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475fb35-7462-48e5-ad7e-372b50ba104a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:13.826487 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d81b9-93fb-4e5f-9c13-dda5e53ddb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:13.827135 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d81b9-93fb-4e5f-9c13-dda5e53ddb15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:13.827411 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d81b9-93fb-4e5f-9c13-dda5e53ddb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:13.827681 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d81b9-93fb-4e5f-9c13-dda5e53ddb15 admin admin] PATH_INFO: `/v3/projects/1e5d152b08a64301995844aa0f3a838a/users/a39b05f09713457da1d08cb685f78eca/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:13.841801 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112d81b9-93fb-4e5f-9c13-dda5e53ddb15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e5d152b08a64301995844aa0f3a838a, user_id=a39b05f09713457da1d08cb685f78eca, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:13.844961 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112d81b9-93fb-4e5f-9c13-dda5e53ddb15 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:13.857726 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1389/2742] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:10:13 2022] PUT /identity/v3/projects/1e5d152b08a64301995844aa0f3a838a/users/a39b05f09713457da1d08cb685f78eca/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:13.868632 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d323524-a30e-43d9-930a-932b697bef43 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:13.868919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d323524-a30e-43d9-930a-932b697bef43 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:13.869193 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d323524-a30e-43d9-930a-932b697bef43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:13.895575 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-8d323524-a30e-43d9-930a-932b697bef43 None None] MFA Rules not processed for user `a39b05f09713457da1d08cb685f78eca`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:13.897573 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-8d323524-a30e-43d9-930a-932b697bef43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:13.961067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1354/2743] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:13.965573 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d81b9-93fb-4e5f-9c13-dda5e53ddb15 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:14.025803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d81b9-93fb-4e5f-9c13-dda5e53ddb15 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:14.031117 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5423291-65c1-47b4-a24b-3b3099f94fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:14.031712 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5423291-65c1-47b4-a24b-3b3099f94fcc admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:14.031994 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5423291-65c1-47b4-a24b-3b3099f94fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:14.032260 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5423291-65c1-47b4-a24b-3b3099f94fcc admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:14.033082 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5423291-65c1-47b4-a24b-3b3099f94fcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:14.034964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5423291-65c1-47b4-a24b-3b3099f94fcc admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:14.062967 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1390/2744] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:10:13 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:10:14.066941 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d323524-a30e-43d9-930a-932b697bef43 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:14.078037 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d323524-a30e-43d9-930a-932b697bef43 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:14.131756 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a472cb-0a73-4d7c-87b5-76fdf023c2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:14.132388 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a472cb-0a73-4d7c-87b5-76fdf023c2bb admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:14.132668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a472cb-0a73-4d7c-87b5-76fdf023c2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:14.132941 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a472cb-0a73-4d7c-87b5-76fdf023c2bb admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:14.133587 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a472cb-0a73-4d7c-87b5-76fdf023c2bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:14.135490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a472cb-0a73-4d7c-87b5-76fdf023c2bb admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:14.157447 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1355/2745] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:10:14 2022] POST /identity/v3/users => generated 298 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:10:14.161499 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5423291-65c1-47b4-a24b-3b3099f94fcc admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:14.171764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5423291-65c1-47b4-a24b-3b3099f94fcc admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:14.176372 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6062e-8fce-40e8-8f31-e65a7940ca36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:14.176969 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6062e-8fce-40e8-8f31-e65a7940ca36 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:14.177248 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6062e-8fce-40e8-8f31-e65a7940ca36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:14.177521 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6062e-8fce-40e8-8f31-e65a7940ca36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:14.178203 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b6062e-8fce-40e8-8f31-e65a7940ca36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:14.179780 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b6062e-8fce-40e8-8f31-e65a7940ca36 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:14.190266 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1391/2746] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:10:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:10:14.194222 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a472cb-0a73-4d7c-87b5-76fdf023c2bb admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:14.204461 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a472cb-0a73-4d7c-87b5-76fdf023c2bb admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:14.209079 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa1f4a-fae3-46c8-b07f-20bf3f23debc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:14.209721 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa1f4a-fae3-46c8-b07f-20bf3f23debc admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:14.210003 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa1f4a-fae3-46c8-b07f-20bf3f23debc admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:14.210273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa1f4a-fae3-46c8-b07f-20bf3f23debc admin admin] PATH_INFO: `/v3/projects/479064d17b924f41b70220706a6eaab0/users/c1d839fc72ee42c68266c6e6f629ebeb/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:14.225640 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa1f4a-fae3-46c8-b07f-20bf3f23debc admin admin] RBAC: Authorizing `identity:create_grant(project_id=479064d17b924f41b70220706a6eaab0, user_id=c1d839fc72ee42c68266c6e6f629ebeb, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:14.228498 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa1f4a-fae3-46c8-b07f-20bf3f23debc admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:14.240658 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1356/2747] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:10:14 2022] PUT /identity/v3/projects/479064d17b924f41b70220706a6eaab0/users/c1d839fc72ee42c68266c6e6f629ebeb/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:14.244685 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6062e-8fce-40e8-8f31-e65a7940ca36 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:14.254901 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6062e-8fce-40e8-8f31-e65a7940ca36 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:14.310980 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ddb73-4ffa-4785-b03b-1ade861f8eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:14.312084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ddb73-4ffa-4785-b03b-1ade861f8eca admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:14.312387 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ddb73-4ffa-4785-b03b-1ade861f8eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:14.312659 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ddb73-4ffa-4785-b03b-1ade861f8eca admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:14.313387 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ddb73-4ffa-4785-b03b-1ade861f8eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:14.314933 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ddb73-4ffa-4785-b03b-1ade861f8eca admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:14.317014 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:14.329125 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1392/2748] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:10:14 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:10:14.330437 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa1f4a-fae3-46c8-b07f-20bf3f23debc admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:14.340967 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa1f4a-fae3-46c8-b07f-20bf3f23debc admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:14.345624 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25687886-fd4a-4c44-b0e6-09ad18f196ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:14.346275 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25687886-fd4a-4c44-b0e6-09ad18f196ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:14.346555 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25687886-fd4a-4c44-b0e6-09ad18f196ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:14.346887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25687886-fd4a-4c44-b0e6-09ad18f196ff admin admin] PATH_INFO: `/v3/projects/479064d17b924f41b70220706a6eaab0/users/c1d839fc72ee42c68266c6e6f629ebeb/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:14.349129 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25687886-fd4a-4c44-b0e6-09ad18f196ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=479064d17b924f41b70220706a6eaab0, user_id=c1d839fc72ee42c68266c6e6f629ebeb, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:14.351835 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25687886-fd4a-4c44-b0e6-09ad18f196ff admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:14.357304 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:14.363773 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1357/2749] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:10:14 2022] PUT /identity/v3/projects/479064d17b924f41b70220706a6eaab0/users/c1d839fc72ee42c68266c6e6f629ebeb/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:14.374386 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939aa171-acf2-4505-b6fd-757a2fc410cc None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:14.374699 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939aa171-acf2-4505-b6fd-757a2fc410cc None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:14.375002 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939aa171-acf2-4505-b6fd-757a2fc410cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:14.402557 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-939aa171-acf2-4505-b6fd-757a2fc410cc None None] MFA Rules not processed for user `c1d839fc72ee42c68266c6e6f629ebeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:14.403755 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-939aa171-acf2-4505-b6fd-757a2fc410cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:14.469713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1393/2750] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:14.480412 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25687886-fd4a-4c44-b0e6-09ad18f196ff admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:14.482076 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-25687886-fd4a-4c44-b0e6-09ad18f196ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:14.537717 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25687886-fd4a-4c44-b0e6-09ad18f196ff admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:14.541289 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ee3fa-11f0-48a1-bd74-a85f0e18dde5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:14.541887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ee3fa-11f0-48a1-bd74-a85f0e18dde5 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:14.542161 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ee3fa-11f0-48a1-bd74-a85f0e18dde5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:14.542430 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ee3fa-11f0-48a1-bd74-a85f0e18dde5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:14.598385 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ee3fa-11f0-48a1-bd74-a85f0e18dde5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:14.600066 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ee3fa-11f0-48a1-bd74-a85f0e18dde5 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:14.609040 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1358/2751] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:10:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:10:18.879569 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939aa171-acf2-4505-b6fd-757a2fc410cc None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:18.889215 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939aa171-acf2-4505-b6fd-757a2fc410cc None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:18.893368 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481ff73-ffef-4301-8653-26a9ed9ca5e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:18.893974 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481ff73-ffef-4301-8653-26a9ed9ca5e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:18.894249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481ff73-ffef-4301-8653-26a9ed9ca5e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:18.894518 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481ff73-ffef-4301-8653-26a9ed9ca5e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:18.903303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481ff73-ffef-4301-8653-26a9ed9ca5e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:18.904790 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481ff73-ffef-4301-8653-26a9ed9ca5e7 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:18.913926 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1394/2752] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:10:18 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:10:23.791095 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ee3fa-11f0-48a1-bd74-a85f0e18dde5 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:23.853557 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ee3fa-11f0-48a1-bd74-a85f0e18dde5 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:23.858091 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea538429-3400-42d4-a507-69550b5d3ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:23.858725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea538429-3400-42d4-a507-69550b5d3ced admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:23.859008 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea538429-3400-42d4-a507-69550b5d3ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:23.859282 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea538429-3400-42d4-a507-69550b5d3ced admin admin] PATH_INFO: `/v3/users/a39b05f09713457da1d08cb685f78eca` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:23.861561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea538429-3400-42d4-a507-69550b5d3ced admin admin] RBAC: Authorizing `identity:delete_user(user_id=a39b05f09713457da1d08cb685f78eca)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:23.863751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea538429-3400-42d4-a507-69550b5d3ced admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:23.913265 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ea538429-3400-42d4-a507-69550b5d3ced admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a39b05f09713457da1d08cb685f78eca'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:23.919605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:23.923360 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ea538429-3400-42d4-a507-69550b5d3ced admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a39b05f09713457da1d08cb685f78eca'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:23.939312 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ea538429-3400-42d4-a507-69550b5d3ced admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a39b05f09713457da1d08cb685f78eca'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:23.940127 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ea538429-3400-42d4-a507-69550b5d3ced admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a39b05f09713457da1d08cb685f78eca'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:23.940749 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-ea538429-3400-42d4-a507-69550b5d3ced admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a39b05f09713457da1d08cb685f78eca'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:23.955527 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1359/2753] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:10:23 2022] DELETE /identity/v3/users/a39b05f09713457da1d08cb685f78eca => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:23.960288 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b55bd8-373f-447d-9813-4f4e1a90cfbc None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:23.960816 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b55bd8-373f-447d-9813-4f4e1a90cfbc None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:23.960911 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b55bd8-373f-447d-9813-4f4e1a90cfbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:23.980202 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d9b55bd8-373f-447d-9813-4f4e1a90cfbc None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:23.981444 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d9b55bd8-373f-447d-9813-4f4e1a90cfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:24.043557 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1395/2754] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:24.053383 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea538429-3400-42d4-a507-69550b5d3ced admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:24.054771 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-ea538429-3400-42d4-a507-69550b5d3ced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:24.112233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea538429-3400-42d4-a507-69550b5d3ced admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:24.115878 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e85ca1-a13c-4cf7-9169-be5046a2a2fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:24.116517 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e85ca1-a13c-4cf7-9169-be5046a2a2fd service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:24.116805 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e85ca1-a13c-4cf7-9169-be5046a2a2fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:24.117070 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e85ca1-a13c-4cf7-9169-be5046a2a2fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:24.128277 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e85ca1-a13c-4cf7-9169-be5046a2a2fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:24.130489 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e85ca1-a13c-4cf7-9169-be5046a2a2fd service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:24.141960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1360/2755] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:10:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 13:10:24.579652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b55bd8-373f-447d-9813-4f4e1a90cfbc None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:24.581176 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d9b55bd8-373f-447d-9813-4f4e1a90cfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:24.593438 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b55bd8-373f-447d-9813-4f4e1a90cfbc None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:24.598114 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87a028-d836-4f0e-bc6b-c06ab665e47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:24.599051 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87a028-d836-4f0e-bc6b-c06ab665e47d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:24.599317 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87a028-d836-4f0e-bc6b-c06ab665e47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:24.599585 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87a028-d836-4f0e-bc6b-c06ab665e47d admin admin] PATH_INFO: `/v3/projects/1e5d152b08a64301995844aa0f3a838a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:24.600875 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a87a028-d836-4f0e-bc6b-c06ab665e47d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e5d152b08a64301995844aa0f3a838a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:24.602817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a87a028-d836-4f0e-bc6b-c06ab665e47d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:24.656472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9a87a028-d836-4f0e-bc6b-c06ab665e47d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e5d152b08a64301995844aa0f3a838a', 'request_id': 'req-9a87a028-d836-4f0e-bc6b-c06ab665e47d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:24.662908 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9a87a028-d836-4f0e-bc6b-c06ab665e47d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e5d152b08a64301995844aa0f3a838a', 'request_id': 'req-9a87a028-d836-4f0e-bc6b-c06ab665e47d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:24.671540 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9a87a028-d836-4f0e-bc6b-c06ab665e47d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e5d152b08a64301995844aa0f3a838a', 'request_id': 'req-9a87a028-d836-4f0e-bc6b-c06ab665e47d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:24.672871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:24.684743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9a87a028-d836-4f0e-bc6b-c06ab665e47d admin admin] The token cache is being invalidate because project 1e5d152b08a64301995844aa0f3a838a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:10:24.685030 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9a87a028-d836-4f0e-bc6b-c06ab665e47d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:24.685687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9a87a028-d836-4f0e-bc6b-c06ab665e47d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:24.687784 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1396/2756] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:10:24 2022] DELETE /identity/v3/projects/1e5d152b08a64301995844aa0f3a838a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:24.691791 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e85ca1-a13c-4cf7-9169-be5046a2a2fd service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:24.701471 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e85ca1-a13c-4cf7-9169-be5046a2a2fd service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:24.763999 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d6b36-ce02-4a16-bee1-fc3f08399238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:24.764683 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d6b36-ce02-4a16-bee1-fc3f08399238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:24.764962 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d6b36-ce02-4a16-bee1-fc3f08399238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:24.765233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d6b36-ce02-4a16-bee1-fc3f08399238 admin admin] PATH_INFO: `/v3/users/c1d839fc72ee42c68266c6e6f629ebeb` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:24.766393 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d6b36-ce02-4a16-bee1-fc3f08399238 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1d839fc72ee42c68266c6e6f629ebeb)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:24.768330 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d6b36-ce02-4a16-bee1-fc3f08399238 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:24.819261 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3b3d6b36-ce02-4a16-bee1-fc3f08399238 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1d839fc72ee42c68266c6e6f629ebeb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:24.826021 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3b3d6b36-ce02-4a16-bee1-fc3f08399238 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1d839fc72ee42c68266c6e6f629ebeb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:24.837660 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3b3d6b36-ce02-4a16-bee1-fc3f08399238 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1d839fc72ee42c68266c6e6f629ebeb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:24.838453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3b3d6b36-ce02-4a16-bee1-fc3f08399238 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1d839fc72ee42c68266c6e6f629ebeb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:24.839056 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3b3d6b36-ce02-4a16-bee1-fc3f08399238 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1d839fc72ee42c68266c6e6f629ebeb'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:24.841541 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:24.856429 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1361/2757] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:10:24 2022] DELETE /identity/v3/users/c1d839fc72ee42c68266c6e6f629ebeb => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:25.220759 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87a028-d836-4f0e-bc6b-c06ab665e47d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:25.222764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9a87a028-d836-4f0e-bc6b-c06ab665e47d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:25.241256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87a028-d836-4f0e-bc6b-c06ab665e47d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:25.293425 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d6b36-ce02-4a16-bee1-fc3f08399238 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:25.294874 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3b3d6b36-ce02-4a16-bee1-fc3f08399238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:25.303069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7e534-2dbc-4b80-bf37-9678d8b5b595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:25.303904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7e534-2dbc-4b80-bf37-9678d8b5b595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:25.304508 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7e534-2dbc-4b80-bf37-9678d8b5b595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:25.304858 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7e534-2dbc-4b80-bf37-9678d8b5b595 admin admin] PATH_INFO: `/v3/users/3bc8a3a20d1044ad8e5e71b49740c2d3` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:25.306883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b7e534-2dbc-4b80-bf37-9678d8b5b595 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bc8a3a20d1044ad8e5e71b49740c2d3)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:25.309053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b7e534-2dbc-4b80-bf37-9678d8b5b595 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:25.358994 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-80b7e534-2dbc-4b80-bf37-9678d8b5b595 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bc8a3a20d1044ad8e5e71b49740c2d3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:25.362833 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d6b36-ce02-4a16-bee1-fc3f08399238 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:25.367654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d4868-c223-4716-92ee-8d553eb07578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:25.368296 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d4868-c223-4716-92ee-8d553eb07578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:25.368590 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d4868-c223-4716-92ee-8d553eb07578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:25.368865 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d4868-c223-4716-92ee-8d553eb07578 admin admin] PATH_INFO: `/v3/projects/479064d17b924f41b70220706a6eaab0` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:25.370005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d4868-c223-4716-92ee-8d553eb07578 admin admin] RBAC: Authorizing `identity:delete_project(project_id=479064d17b924f41b70220706a6eaab0)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:25.372017 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d4868-c223-4716-92ee-8d553eb07578 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:25.372862 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-80b7e534-2dbc-4b80-bf37-9678d8b5b595 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bc8a3a20d1044ad8e5e71b49740c2d3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:25.373719 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-80b7e534-2dbc-4b80-bf37-9678d8b5b595 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bc8a3a20d1044ad8e5e71b49740c2d3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:25.374655 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-80b7e534-2dbc-4b80-bf37-9678d8b5b595 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bc8a3a20d1044ad8e5e71b49740c2d3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:25.382291 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-80b7e534-2dbc-4b80-bf37-9678d8b5b595 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bc8a3a20d1044ad8e5e71b49740c2d3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:25.395406 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1397/2758] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:10:25 2022] DELETE /identity/v3/users/3bc8a3a20d1044ad8e5e71b49740c2d3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:25.421694 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e93d4868-c223-4716-92ee-8d553eb07578 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '479064d17b924f41b70220706a6eaab0', 'request_id': 'req-e93d4868-c223-4716-92ee-8d553eb07578'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:25.427071 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e93d4868-c223-4716-92ee-8d553eb07578 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '479064d17b924f41b70220706a6eaab0', 'request_id': 'req-e93d4868-c223-4716-92ee-8d553eb07578'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:25.434818 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e93d4868-c223-4716-92ee-8d553eb07578 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '479064d17b924f41b70220706a6eaab0', 'request_id': 'req-e93d4868-c223-4716-92ee-8d553eb07578'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:25.446987 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e93d4868-c223-4716-92ee-8d553eb07578 admin admin] The token cache is being invalidate because project 479064d17b924f41b70220706a6eaab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:10:25.447275 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e93d4868-c223-4716-92ee-8d553eb07578 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:25.447953 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e93d4868-c223-4716-92ee-8d553eb07578 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:25.450155 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1362/2759] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:10:25 2022] DELETE /identity/v3/projects/479064d17b924f41b70220706a6eaab0 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:10:25.456547 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1068747-11b7-4de1-90ed-c6d4e2a6338d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:25.456839 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1068747-11b7-4de1-90ed-c6d4e2a6338d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:25.457115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1068747-11b7-4de1-90ed-c6d4e2a6338d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:25.477831 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a1068747-11b7-4de1-90ed-c6d4e2a6338d None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:25.479083 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a1068747-11b7-4de1-90ed-c6d4e2a6338d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:25.539155 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1398/2760] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:25.547459 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d119d-bb98-4937-99dc-6e09f52e6630 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:25.547743 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d119d-bb98-4937-99dc-6e09f52e6630 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:25.548016 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d119d-bb98-4937-99dc-6e09f52e6630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:25.567568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-254d119d-bb98-4937-99dc-6e09f52e6630 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:25.568792 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-254d119d-bb98-4937-99dc-6e09f52e6630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:25.577171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1068747-11b7-4de1-90ed-c6d4e2a6338d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:25.578988 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a1068747-11b7-4de1-90ed-c6d4e2a6338d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:25.581155 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1363/2761] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:25.585206 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d119d-bb98-4937-99dc-6e09f52e6630 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:25.594606 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d119d-bb98-4937-99dc-6e09f52e6630 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:25.598180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1068747-11b7-4de1-90ed-c6d4e2a6338d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:25.601048 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb888a-f00c-42fd-9997-ba914bf144b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:25.601728 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb888a-f00c-42fd-9997-ba914bf144b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:25.602030 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb888a-f00c-42fd-9997-ba914bf144b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:25.602335 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb888a-f00c-42fd-9997-ba914bf144b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:25.603034 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb888a-f00c-42fd-9997-ba914bf144b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:10:25.603717 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb888a-f00c-42fd-9997-ba914bf144b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:25.604960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb888a-f00c-42fd-9997-ba914bf144b4 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:25.616917 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c09f3-ce6b-47ae-b224-fd85b5a0b269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:25.616917 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c09f3-ce6b-47ae-b224-fd85b5a0b269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:25.616917 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c09f3-ce6b-47ae-b224-fd85b5a0b269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:25.617264 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c09f3-ce6b-47ae-b224-fd85b5a0b269 admin admin] PATH_INFO: `/v3/projects/8f8ad8bc9879474b8de3383986e5bff0` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:25.618304 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c09f3-ce6b-47ae-b224-fd85b5a0b269 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f8ad8bc9879474b8de3383986e5bff0)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:25.622369 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1364/2762] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 13:10:25.622491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c09f3-ce6b-47ae-b224-fd85b5a0b269 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:25.623767 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb888a-f00c-42fd-9997-ba914bf144b4 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:25.629692 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:25.634891 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb888a-f00c-42fd-9997-ba914bf144b4 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:25.641068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5626605-4185-4841-af1a-12698bf2852d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:25.641792 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5626605-4185-4841-af1a-12698bf2852d admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:25.642106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5626605-4185-4841-af1a-12698bf2852d admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:25.642406 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5626605-4185-4841-af1a-12698bf2852d admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:25.643071 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5626605-4185-4841-af1a-12698bf2852d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:25.645001 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5626605-4185-4841-af1a-12698bf2852d admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:25.678565 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1365/2763] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:10:25 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:10:25.682302 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5626605-4185-4841-af1a-12698bf2852d admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:25.683280 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ae0c09f3-ce6b-47ae-b224-fd85b5a0b269 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f8ad8bc9879474b8de3383986e5bff0', 'request_id': 'req-ae0c09f3-ce6b-47ae-b224-fd85b5a0b269'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:25.689228 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ae0c09f3-ce6b-47ae-b224-fd85b5a0b269 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f8ad8bc9879474b8de3383986e5bff0', 'request_id': 'req-ae0c09f3-ce6b-47ae-b224-fd85b5a0b269'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:25.696740 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ae0c09f3-ce6b-47ae-b224-fd85b5a0b269 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f8ad8bc9879474b8de3383986e5bff0', 'request_id': 'req-ae0c09f3-ce6b-47ae-b224-fd85b5a0b269'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:25.708993 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ae0c09f3-ce6b-47ae-b224-fd85b5a0b269 admin admin] The token cache is being invalidate because project 8f8ad8bc9879474b8de3383986e5bff0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:10:25.709283 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ae0c09f3-ce6b-47ae-b224-fd85b5a0b269 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:25.709905 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ae0c09f3-ce6b-47ae-b224-fd85b5a0b269 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:25.711533 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1399/2764] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:10:25 2022] DELETE /identity/v3/projects/8f8ad8bc9879474b8de3383986e5bff0 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:10:25.715273 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c09f3-ce6b-47ae-b224-fd85b5a0b269 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:25.716587 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ae0c09f3-ce6b-47ae-b224-fd85b5a0b269 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:25.728086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:25.748222 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5626605-4185-4841-af1a-12698bf2852d admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:25.753730 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6b6cf-bcf6-4aed-871d-82c4b1b9e5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:25.754343 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6b6cf-bcf6-4aed-871d-82c4b1b9e5a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:25.754626 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6b6cf-bcf6-4aed-871d-82c4b1b9e5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:25.754905 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c6b6cf-bcf6-4aed-871d-82c4b1b9e5a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:25.755566 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c6b6cf-bcf6-4aed-871d-82c4b1b9e5a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:25.757624 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c6b6cf-bcf6-4aed-871d-82c4b1b9e5a4 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:25.778807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1366/2765] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:10:25 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:10:25.782382 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c09f3-ce6b-47ae-b224-fd85b5a0b269 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:25.782989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6b6cf-bcf6-4aed-871d-82c4b1b9e5a4 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:25.784954 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a3c6b6cf-bcf6-4aed-871d-82c4b1b9e5a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:25.787167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c6baa-7b30-4873-bc31-d18f8e54571b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:25.787813 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c6baa-7b30-4873-bc31-d18f8e54571b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:25.788735 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c6baa-7b30-4873-bc31-d18f8e54571b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:25.789012 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c6baa-7b30-4873-bc31-d18f8e54571b admin admin] PATH_INFO: `/v3/users/97c00b665ccc41569089e3d66d911091` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:25.790337 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108c6baa-7b30-4873-bc31-d18f8e54571b admin admin] RBAC: Authorizing `identity:delete_user(user_id=97c00b665ccc41569089e3d66d911091)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:25.792273 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108c6baa-7b30-4873-bc31-d18f8e54571b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:25.797607 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c6b6cf-bcf6-4aed-871d-82c4b1b9e5a4 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:25.803139 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a946b6-fca2-4184-8664-8b1df80fb0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:25.804081 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a946b6-fca2-4184-8664-8b1df80fb0f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:25.804582 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a946b6-fca2-4184-8664-8b1df80fb0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:25.805038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a946b6-fca2-4184-8664-8b1df80fb0f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:25.806283 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a946b6-fca2-4184-8664-8b1df80fb0f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:25.808191 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a946b6-fca2-4184-8664-8b1df80fb0f2 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:25.819648 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1367/2766] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 13:10:25.823756 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a946b6-fca2-4184-8664-8b1df80fb0f2 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:25.834336 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a946b6-fca2-4184-8664-8b1df80fb0f2 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:25.838234 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-108c6baa-7b30-4873-bc31-d18f8e54571b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97c00b665ccc41569089e3d66d911091'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:25.839450 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d2f4f-7fc4-44b6-9566-8915e697db48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:25.840047 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d2f4f-7fc4-44b6-9566-8915e697db48 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:25.840359 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d2f4f-7fc4-44b6-9566-8915e697db48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:25.840659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d2f4f-7fc4-44b6-9566-8915e697db48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:25.841329 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d2f4f-7fc4-44b6-9566-8915e697db48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:25.842941 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d2f4f-7fc4-44b6-9566-8915e697db48 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:25.851108 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-108c6baa-7b30-4873-bc31-d18f8e54571b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97c00b665ccc41569089e3d66d911091'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:25.851917 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-108c6baa-7b30-4873-bc31-d18f8e54571b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97c00b665ccc41569089e3d66d911091'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:25.852559 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-108c6baa-7b30-4873-bc31-d18f8e54571b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97c00b665ccc41569089e3d66d911091'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:25.853581 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1368/2767] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:10:25.857578 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d2f4f-7fc4-44b6-9566-8915e697db48 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:25.859554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-de1d2f4f-7fc4-44b6-9566-8915e697db48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:25.860908 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-108c6baa-7b30-4873-bc31-d18f8e54571b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97c00b665ccc41569089e3d66d911091'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:25.872672 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d2f4f-7fc4-44b6-9566-8915e697db48 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:25.874146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1400/2768] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:10:25 2022] DELETE /identity/v3/users/97c00b665ccc41569089e3d66d911091 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:10:25.931353 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f75df-e72f-43b8-8de2-ca1f906ba2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:25.932026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f75df-e72f-43b8-8de2-ca1f906ba2df admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:25.932334 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f75df-e72f-43b8-8de2-ca1f906ba2df admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:25.932733 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f75df-e72f-43b8-8de2-ca1f906ba2df admin admin] PATH_INFO: `/v3/projects/ae11ccde91424029be1418addab91b6f/users/96d799aebfca4c9699844a6d8e92b0b8/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:25.948111 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99f75df-e72f-43b8-8de2-ca1f906ba2df admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae11ccde91424029be1418addab91b6f, user_id=96d799aebfca4c9699844a6d8e92b0b8, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:25.950816 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99f75df-e72f-43b8-8de2-ca1f906ba2df admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:25.964646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1369/2769] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:10:25 2022] PUT /identity/v3/projects/ae11ccde91424029be1418addab91b6f/users/96d799aebfca4c9699844a6d8e92b0b8/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:10:25.975893 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8dba0-ffb8-4def-bce0-f0240faa2eb9 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:25.976217 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8dba0-ffb8-4def-bce0-f0240faa2eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:25.976530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8dba0-ffb8-4def-bce0-f0240faa2eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:26.008658 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0cc8dba0-ffb8-4def-bce0-f0240faa2eb9 None None] MFA Rules not processed for user `96d799aebfca4c9699844a6d8e92b0b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:26.009891 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0cc8dba0-ffb8-4def-bce0-f0240faa2eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:26.013917 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:26.050328 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f75df-e72f-43b8-8de2-ca1f906ba2df admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:26.052165 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d99f75df-e72f-43b8-8de2-ca1f906ba2df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:26.086851 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1401/2770] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:26.097540 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8dba0-ffb8-4def-bce0-f0240faa2eb9 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:26.099154 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0cc8dba0-ffb8-4def-bce0-f0240faa2eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:26.125371 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f75df-e72f-43b8-8de2-ca1f906ba2df admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:26.130040 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc352769-7a9a-4372-94bd-e761d79abe78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:26.130775 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc352769-7a9a-4372-94bd-e761d79abe78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:26.131151 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc352769-7a9a-4372-94bd-e761d79abe78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:26.131526 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc352769-7a9a-4372-94bd-e761d79abe78 admin admin] PATH_INFO: `/v3/projects/a718128833e24d10ada84a5573181680` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:26.132794 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc352769-7a9a-4372-94bd-e761d79abe78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a718128833e24d10ada84a5573181680)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:26.134726 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc352769-7a9a-4372-94bd-e761d79abe78 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:26.157695 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8dba0-ffb8-4def-bce0-f0240faa2eb9 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:26.161665 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b131a9-cecc-40d8-bb83-c91027f2a28d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:26.162326 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b131a9-cecc-40d8-bb83-c91027f2a28d service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:26.162695 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b131a9-cecc-40d8-bb83-c91027f2a28d service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:26.162968 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b131a9-cecc-40d8-bb83-c91027f2a28d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:26.172434 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b131a9-cecc-40d8-bb83-c91027f2a28d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:26.174071 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b131a9-cecc-40d8-bb83-c91027f2a28d service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:26.174233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:26.183902 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1402/2771] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:10:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 13:10:26.184676 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-cc352769-7a9a-4372-94bd-e761d79abe78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a718128833e24d10ada84a5573181680', 'request_id': 'req-cc352769-7a9a-4372-94bd-e761d79abe78'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:26.191184 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-cc352769-7a9a-4372-94bd-e761d79abe78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a718128833e24d10ada84a5573181680', 'request_id': 'req-cc352769-7a9a-4372-94bd-e761d79abe78'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:26.199092 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-cc352769-7a9a-4372-94bd-e761d79abe78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a718128833e24d10ada84a5573181680', 'request_id': 'req-cc352769-7a9a-4372-94bd-e761d79abe78'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:26.210419 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-cc352769-7a9a-4372-94bd-e761d79abe78 admin admin] The token cache is being invalidate because project a718128833e24d10ada84a5573181680 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:10:26.210708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-cc352769-7a9a-4372-94bd-e761d79abe78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:26.211355 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-cc352769-7a9a-4372-94bd-e761d79abe78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:26.213438 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1370/2772] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:10:26 2022] DELETE /identity/v3/projects/a718128833e24d10ada84a5573181680 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:10:26.218497 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d990102-8ed8-4100-84a5-de7c52d5f9d1 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:26.218789 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d990102-8ed8-4100-84a5-de7c52d5f9d1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:26.219066 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d990102-8ed8-4100-84a5-de7c52d5f9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:26.239452 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-7d990102-8ed8-4100-84a5-de7c52d5f9d1 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:26.240693 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7d990102-8ed8-4100-84a5-de7c52d5f9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:26.301218 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1403/2773] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:26.309301 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71abf0-a237-4543-84da-695572ccfec5 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:26.309695 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71abf0-a237-4543-84da-695572ccfec5 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:26.310005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71abf0-a237-4543-84da-695572ccfec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:26.331636 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3f71abf0-a237-4543-84da-695572ccfec5 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:26.332893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3f71abf0-a237-4543-84da-695572ccfec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:26.340894 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1371/2774] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:26.345710 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d990102-8ed8-4100-84a5-de7c52d5f9d1 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:26.356139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d990102-8ed8-4100-84a5-de7c52d5f9d1 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:26.361509 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c6d03-6971-4a66-bd5d-afd493df31c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:26.362122 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c6d03-6971-4a66-bd5d-afd493df31c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:26.362400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c6d03-6971-4a66-bd5d-afd493df31c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:26.362670 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c6d03-6971-4a66-bd5d-afd493df31c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:26.363361 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c6d03-6971-4a66-bd5d-afd493df31c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:10:26.363657 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c6d03-6971-4a66-bd5d-afd493df31c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:26.365293 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c6d03-6971-4a66-bd5d-afd493df31c8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:26.377336 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1404/2775] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:10:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 13:10:26.381665 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71abf0-a237-4543-84da-695572ccfec5 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:26.392102 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71abf0-a237-4543-84da-695572ccfec5 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:26.397257 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c05737-88f0-44ad-b659-22f4f6694594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:26.397844 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c05737-88f0-44ad-b659-22f4f6694594 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:26.398141 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c05737-88f0-44ad-b659-22f4f6694594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:26.398396 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c05737-88f0-44ad-b659-22f4f6694594 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:26.399060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c05737-88f0-44ad-b659-22f4f6694594 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:26.401051 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c05737-88f0-44ad-b659-22f4f6694594 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:26.436344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1372/2776] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:10:26 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:10:26.439827 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c6d03-6971-4a66-bd5d-afd493df31c8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:26.450892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c6d03-6971-4a66-bd5d-afd493df31c8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:26.499403 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:26.524851 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbca508-f05d-489a-8a1d-7a6b61da3717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:26.525468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbca508-f05d-489a-8a1d-7a6b61da3717 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:26.525750 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbca508-f05d-489a-8a1d-7a6b61da3717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:26.526023 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbca508-f05d-489a-8a1d-7a6b61da3717 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:26.526683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbca508-f05d-489a-8a1d-7a6b61da3717 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:26.528604 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbca508-f05d-489a-8a1d-7a6b61da3717 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:26.551021 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1405/2777] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:10:26 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:10:26.555207 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c05737-88f0-44ad-b659-22f4f6694594 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:26.566007 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c05737-88f0-44ad-b659-22f4f6694594 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:26.570639 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86abd25e-11fa-4228-a72b-b1b9b9421bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:26.571249 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abd25e-11fa-4228-a72b-b1b9b9421bcd admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:26.571535 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abd25e-11fa-4228-a72b-b1b9b9421bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:26.571840 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abd25e-11fa-4228-a72b-b1b9b9421bcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:26.572644 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86abd25e-11fa-4228-a72b-b1b9b9421bcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:26.574590 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86abd25e-11fa-4228-a72b-b1b9b9421bcd admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:26.585411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1373/2778] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:10:26.589170 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbca508-f05d-489a-8a1d-7a6b61da3717 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:26.599808 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbca508-f05d-489a-8a1d-7a6b61da3717 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:26.604570 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61385f3-3b55-4425-b3f9-95cac9765707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:26.605177 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61385f3-3b55-4425-b3f9-95cac9765707 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:26.605461 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61385f3-3b55-4425-b3f9-95cac9765707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:26.605739 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61385f3-3b55-4425-b3f9-95cac9765707 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:26.606429 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61385f3-3b55-4425-b3f9-95cac9765707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:26.608596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61385f3-3b55-4425-b3f9-95cac9765707 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:26.619091 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1406/2779] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:10:26.622916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86abd25e-11fa-4228-a72b-b1b9b9421bcd admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:26.632713 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86abd25e-11fa-4228-a72b-b1b9b9421bcd admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:26.637175 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68124d31-aac8-4b85-90e4-e80c3242d4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:26.637818 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68124d31-aac8-4b85-90e4-e80c3242d4b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:26.638097 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68124d31-aac8-4b85-90e4-e80c3242d4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:26.638370 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68124d31-aac8-4b85-90e4-e80c3242d4b7 admin admin] PATH_INFO: `/v3/projects/dea9d1c28eee4b0ca2a1af39ca7d3576/users/79af81cd177248979f4c73c93a7baa2c/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:26.653204 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68124d31-aac8-4b85-90e4-e80c3242d4b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dea9d1c28eee4b0ca2a1af39ca7d3576, user_id=79af81cd177248979f4c73c93a7baa2c, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:26.656399 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68124d31-aac8-4b85-90e4-e80c3242d4b7 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:26.669496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1374/2780] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:10:26 2022] PUT /identity/v3/projects/dea9d1c28eee4b0ca2a1af39ca7d3576/users/79af81cd177248979f4c73c93a7baa2c/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:26.681223 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f6271-cb38-41d4-9373-69bc6a89a70c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:26.681552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f6271-cb38-41d4-9373-69bc6a89a70c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:26.681807 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f6271-cb38-41d4-9373-69bc6a89a70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:26.712769 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ed4f6271-cb38-41d4-9373-69bc6a89a70c None None] MFA Rules not processed for user `79af81cd177248979f4c73c93a7baa2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:26.713981 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ed4f6271-cb38-41d4-9373-69bc6a89a70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:26.784608 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1407/2781] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:26.796067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68124d31-aac8-4b85-90e4-e80c3242d4b7 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:26.797947 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-68124d31-aac8-4b85-90e4-e80c3242d4b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:26.856162 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68124d31-aac8-4b85-90e4-e80c3242d4b7 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:26.859954 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79e2f7-0d01-4f4c-ae1a-81e0c450e94b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7610d6289efb44599d0cee03b93f9938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:26.860631 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79e2f7-0d01-4f4c-ae1a-81e0c450e94b service glance] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:26.860906 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79e2f7-0d01-4f4c-ae1a-81e0c450e94b service glance] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:26.861176 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79e2f7-0d01-4f4c-ae1a-81e0c450e94b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:26.869678 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79e2f7-0d01-4f4c-ae1a-81e0c450e94b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:26.871094 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79e2f7-0d01-4f4c-ae1a-81e0c450e94b service glance] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:26.880069 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1375/2782] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:10:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:10:36.781293 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=50,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:36.875885 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:DELETE=5,keystone:SELECT=64,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:37.672669 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f6271-cb38-41d4-9373-69bc6a89a70c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:37.674189 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ed4f6271-cb38-41d4-9373-69bc6a89a70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:37.740468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f6271-cb38-41d4-9373-69bc6a89a70c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:37.745014 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94038f-cf5d-4671-a29e-0f6449c6f1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:37.745650 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94038f-cf5d-4671-a29e-0f6449c6f1b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:37.745928 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94038f-cf5d-4671-a29e-0f6449c6f1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:37.746202 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94038f-cf5d-4671-a29e-0f6449c6f1b7 admin admin] PATH_INFO: `/v3/users/96d799aebfca4c9699844a6d8e92b0b8` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:37.747346 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db94038f-cf5d-4671-a29e-0f6449c6f1b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96d799aebfca4c9699844a6d8e92b0b8)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:37.749295 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db94038f-cf5d-4671-a29e-0f6449c6f1b7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:37.800074 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-db94038f-cf5d-4671-a29e-0f6449c6f1b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96d799aebfca4c9699844a6d8e92b0b8'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:37.813293 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-db94038f-cf5d-4671-a29e-0f6449c6f1b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96d799aebfca4c9699844a6d8e92b0b8'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:37.814041 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-db94038f-cf5d-4671-a29e-0f6449c6f1b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96d799aebfca4c9699844a6d8e92b0b8'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:37.814663 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-db94038f-cf5d-4671-a29e-0f6449c6f1b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96d799aebfca4c9699844a6d8e92b0b8'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:37.823269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-db94038f-cf5d-4671-a29e-0f6449c6f1b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96d799aebfca4c9699844a6d8e92b0b8'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:37.836165 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1408/2783] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:10:37 2022] DELETE /identity/v3/users/96d799aebfca4c9699844a6d8e92b0b8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:37.840618 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64d2fc-b5b1-48fd-ad1f-f7106890a133 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:37.840915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64d2fc-b5b1-48fd-ad1f-f7106890a133 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:37.841163 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64d2fc-b5b1-48fd-ad1f-f7106890a133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:37.861292 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-3f64d2fc-b5b1-48fd-ad1f-f7106890a133 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:37.862499 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3f64d2fc-b5b1-48fd-ad1f-f7106890a133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:37.923720 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1376/2784] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:37.933354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94038f-cf5d-4671-a29e-0f6449c6f1b7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:37.934729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-db94038f-cf5d-4671-a29e-0f6449c6f1b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:37.993171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94038f-cf5d-4671-a29e-0f6449c6f1b7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:37.999138 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea9b58-4e28-4669-9a93-14f34744d1c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:37.999770 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea9b58-4e28-4669-9a93-14f34744d1c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:38.000056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea9b58-4e28-4669-9a93-14f34744d1c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:38.000355 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea9b58-4e28-4669-9a93-14f34744d1c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:38.010527 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea9b58-4e28-4669-9a93-14f34744d1c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:38.011976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea9b58-4e28-4669-9a93-14f34744d1c7 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:38.021894 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1409/2785] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:10:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:10:38.295726 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f64d2fc-b5b1-48fd-ad1f-f7106890a133 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:38.297369 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3f64d2fc-b5b1-48fd-ad1f-f7106890a133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:38.314829 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f64d2fc-b5b1-48fd-ad1f-f7106890a133 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:38.321835 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b849c8a-7feb-4258-8586-5f65f3656c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:38.322653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b849c8a-7feb-4258-8586-5f65f3656c7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:38.322940 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b849c8a-7feb-4258-8586-5f65f3656c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:38.323215 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b849c8a-7feb-4258-8586-5f65f3656c7f admin admin] PATH_INFO: `/v3/projects/ae11ccde91424029be1418addab91b6f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:38.324448 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b849c8a-7feb-4258-8586-5f65f3656c7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae11ccde91424029be1418addab91b6f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:38.326930 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b849c8a-7feb-4258-8586-5f65f3656c7f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:38.377424 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7b849c8a-7feb-4258-8586-5f65f3656c7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae11ccde91424029be1418addab91b6f', 'request_id': 'req-7b849c8a-7feb-4258-8586-5f65f3656c7f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:38.382959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7b849c8a-7feb-4258-8586-5f65f3656c7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae11ccde91424029be1418addab91b6f', 'request_id': 'req-7b849c8a-7feb-4258-8586-5f65f3656c7f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:38.390713 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7b849c8a-7feb-4258-8586-5f65f3656c7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae11ccde91424029be1418addab91b6f', 'request_id': 'req-7b849c8a-7feb-4258-8586-5f65f3656c7f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:38.402663 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7b849c8a-7feb-4258-8586-5f65f3656c7f admin admin] The token cache is being invalidate because project ae11ccde91424029be1418addab91b6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:10:38.402949 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7b849c8a-7feb-4258-8586-5f65f3656c7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:38.403586 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-7b849c8a-7feb-4258-8586-5f65f3656c7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:38.405269 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1377/2786] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:10:38 2022] DELETE /identity/v3/projects/ae11ccde91424029be1418addab91b6f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:38.410230 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77494ea6-752d-4eff-9ab6-b73b457cfb61 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:38.410552 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77494ea6-752d-4eff-9ab6-b73b457cfb61 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:38.410834 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77494ea6-752d-4eff-9ab6-b73b457cfb61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:38.430253 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-77494ea6-752d-4eff-9ab6-b73b457cfb61 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:38.431393 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-77494ea6-752d-4eff-9ab6-b73b457cfb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:38.448856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:38.493596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1410/2787] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:38.503638 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f213e-9d5e-4e1d-a0d6-b8752a156bcd None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:38.503919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f213e-9d5e-4e1d-a0d6-b8752a156bcd None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:38.504191 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f213e-9d5e-4e1d-a0d6-b8752a156bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:38.524371 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-747f213e-9d5e-4e1d-a0d6-b8752a156bcd None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:38.525608 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-747f213e-9d5e-4e1d-a0d6-b8752a156bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:38.534496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1378/2788] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 13:10:38.538750 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77494ea6-752d-4eff-9ab6-b73b457cfb61 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:38.548738 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77494ea6-752d-4eff-9ab6-b73b457cfb61 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:38.554103 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d9ba5-2bed-4155-a7f9-8018a42cc002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:38.554734 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d9ba5-2bed-4155-a7f9-8018a42cc002 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:38.555013 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d9ba5-2bed-4155-a7f9-8018a42cc002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:38.555284 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d9ba5-2bed-4155-a7f9-8018a42cc002 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:38.555984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d9ba5-2bed-4155-a7f9-8018a42cc002 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:10:38.556285 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d9ba5-2bed-4155-a7f9-8018a42cc002 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:38.557910 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d9ba5-2bed-4155-a7f9-8018a42cc002 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:38.570327 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1411/2789] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 13:10:38.574675 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f213e-9d5e-4e1d-a0d6-b8752a156bcd None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:38.584883 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f213e-9d5e-4e1d-a0d6-b8752a156bcd None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:38.590043 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e0a79-dfc2-48cd-8b60-a5c9e0f9dc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:38.590669 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e0a79-dfc2-48cd-8b60-a5c9e0f9dc8c admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:38.590949 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e0a79-dfc2-48cd-8b60-a5c9e0f9dc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:38.591275 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e0a79-dfc2-48cd-8b60-a5c9e0f9dc8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:38.591953 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e0a79-dfc2-48cd-8b60-a5c9e0f9dc8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:38.593912 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e0a79-dfc2-48cd-8b60-a5c9e0f9dc8c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:38.622412 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1379/2790] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:10:38 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:10:38.627075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d9ba5-2bed-4155-a7f9-8018a42cc002 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:38.636923 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d9ba5-2bed-4155-a7f9-8018a42cc002 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:38.695064 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a09e74-9ec2-407e-9eaa-2c31639f857a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:38.695592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a09e74-9ec2-407e-9eaa-2c31639f857a admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:38.695866 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a09e74-9ec2-407e-9eaa-2c31639f857a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:38.696141 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a09e74-9ec2-407e-9eaa-2c31639f857a admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:38.696820 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a09e74-9ec2-407e-9eaa-2c31639f857a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:38.698704 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a09e74-9ec2-407e-9eaa-2c31639f857a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:38.719226 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1412/2791] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:10:38 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:10:38.723550 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e0a79-dfc2-48cd-8b60-a5c9e0f9dc8c admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:38.734495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e0a79-dfc2-48cd-8b60-a5c9e0f9dc8c admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:38.739151 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6c3da-3cb9-4632-9ede-ee58a3d4e4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:38.739755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6c3da-3cb9-4632-9ede-ee58a3d4e4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:38.740023 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6c3da-3cb9-4632-9ede-ee58a3d4e4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:38.740292 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6c3da-3cb9-4632-9ede-ee58a3d4e4b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:38.741012 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c6c3da-3cb9-4632-9ede-ee58a3d4e4b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:38.742602 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c6c3da-3cb9-4632-9ede-ee58a3d4e4b5 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:38.753701 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1380/2792] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:10:38.757803 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a09e74-9ec2-407e-9eaa-2c31639f857a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:38.767615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a09e74-9ec2-407e-9eaa-2c31639f857a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:38.772120 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e7c7a-2a1d-4a2b-8502-63ee07688034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:38.772756 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e7c7a-2a1d-4a2b-8502-63ee07688034 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:38.773036 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e7c7a-2a1d-4a2b-8502-63ee07688034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:38.773307 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e7c7a-2a1d-4a2b-8502-63ee07688034 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:38.773989 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e7c7a-2a1d-4a2b-8502-63ee07688034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:38.775558 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e7c7a-2a1d-4a2b-8502-63ee07688034 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:38.788318 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1413/2793] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:10:38.792806 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6c3da-3cb9-4632-9ede-ee58a3d4e4b5 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:38.803417 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6c3da-3cb9-4632-9ede-ee58a3d4e4b5 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:38.807976 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389da096-bb24-4ccd-b70c-a32ec5ed5427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:38.808667 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389da096-bb24-4ccd-b70c-a32ec5ed5427 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:38.808946 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389da096-bb24-4ccd-b70c-a32ec5ed5427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:38.809219 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389da096-bb24-4ccd-b70c-a32ec5ed5427 admin admin] PATH_INFO: `/v3/projects/b8ba2e48caca474f860946df0c9d0baa/users/4f71cfae23f74c96b1cf3329804f369e/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:38.824549 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389da096-bb24-4ccd-b70c-a32ec5ed5427 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8ba2e48caca474f860946df0c9d0baa, user_id=4f71cfae23f74c96b1cf3329804f369e, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:38.827229 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389da096-bb24-4ccd-b70c-a32ec5ed5427 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:38.843881 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1381/2794] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:10:38 2022] PUT /identity/v3/projects/b8ba2e48caca474f860946df0c9d0baa/users/4f71cfae23f74c96b1cf3329804f369e/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:38.849488 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804e0b2-972b-43b1-a4dd-5ed444e6f8f2 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:38.850491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804e0b2-972b-43b1-a4dd-5ed444e6f8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:38.850914 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804e0b2-972b-43b1-a4dd-5ed444e6f8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:38.871350 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6804e0b2-972b-43b1-a4dd-5ed444e6f8f2 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:38.872582 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6804e0b2-972b-43b1-a4dd-5ed444e6f8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:38.938689 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1414/2795] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:38.949711 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389da096-bb24-4ccd-b70c-a32ec5ed5427 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:38.951397 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-389da096-bb24-4ccd-b70c-a32ec5ed5427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:38.975254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:39.024529 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389da096-bb24-4ccd-b70c-a32ec5ed5427 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:39.028269 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086514fd-1fe7-4082-ac4c-dac4002a268d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:39.028951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086514fd-1fe7-4082-ac4c-dac4002a268d service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:39.029776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086514fd-1fe7-4082-ac4c-dac4002a268d service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:39.030046 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086514fd-1fe7-4082-ac4c-dac4002a268d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:39.039835 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086514fd-1fe7-4082-ac4c-dac4002a268d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:39.041318 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086514fd-1fe7-4082-ac4c-dac4002a268d service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:39.052422 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1382/2796] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:10:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:10:39.691172 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d18c7-6bdf-49d8-8c3e-186bee331a55 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:39.691494 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d18c7-6bdf-49d8-8c3e-186bee331a55 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:39.691772 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d18c7-6bdf-49d8-8c3e-186bee331a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:39.721976 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-710d18c7-6bdf-49d8-8c3e-186bee331a55 None None] MFA Rules not processed for user `4f71cfae23f74c96b1cf3329804f369e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:39.723466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-710d18c7-6bdf-49d8-8c3e-186bee331a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:39.740395 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:39.795664 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1415/2797] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:39.806923 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086514fd-1fe7-4082-ac4c-dac4002a268d service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:39.808541 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-086514fd-1fe7-4082-ac4c-dac4002a268d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:39.866486 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086514fd-1fe7-4082-ac4c-dac4002a268d service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:39.870523 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fbc2db-8194-4cdb-acec-9f6f3ed5d6e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:39.871552 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbc2db-8194-4cdb-acec-9f6f3ed5d6e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:39.871808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbc2db-8194-4cdb-acec-9f6f3ed5d6e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:39.872157 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbc2db-8194-4cdb-acec-9f6f3ed5d6e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:39.882268 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fbc2db-8194-4cdb-acec-9f6f3ed5d6e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:39.883737 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fbc2db-8194-4cdb-acec-9f6f3ed5d6e0 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:39.893421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1383/2798] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:10:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:10:42.299527 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441cd0c-61fc-472f-8baf-fccfdc59fb49 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:42.300061 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441cd0c-61fc-472f-8baf-fccfdc59fb49 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:42.300061 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441cd0c-61fc-472f-8baf-fccfdc59fb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:42.318254 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d441cd0c-61fc-472f-8baf-fccfdc59fb49 None None] MFA Rules not processed for user `4f71cfae23f74c96b1cf3329804f369e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:42.319667 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d441cd0c-61fc-472f-8baf-fccfdc59fb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:42.327102 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1416/2799] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:49.889001 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=43 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:52.313127 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=22 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:52.869297 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fbc2db-8194-4cdb-acec-9f6f3ed5d6e0 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:52.870835 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a4fbc2db-8194-4cdb-acec-9f6f3ed5d6e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:52.886402 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fbc2db-8194-4cdb-acec-9f6f3ed5d6e0 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:52.891596 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1acb7a1-2f6a-4e9e-91f0-52c0b33fcb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:52.892242 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acb7a1-2f6a-4e9e-91f0-52c0b33fcb68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:52.892551 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acb7a1-2f6a-4e9e-91f0-52c0b33fcb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:52.892831 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acb7a1-2f6a-4e9e-91f0-52c0b33fcb68 admin admin] PATH_INFO: `/v3/users/79af81cd177248979f4c73c93a7baa2c` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:52.894059 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1acb7a1-2f6a-4e9e-91f0-52c0b33fcb68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79af81cd177248979f4c73c93a7baa2c)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:52.896233 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1acb7a1-2f6a-4e9e-91f0-52c0b33fcb68 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:52.966236 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-b1acb7a1-2f6a-4e9e-91f0-52c0b33fcb68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79af81cd177248979f4c73c93a7baa2c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:52.974445 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-b1acb7a1-2f6a-4e9e-91f0-52c0b33fcb68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79af81cd177248979f4c73c93a7baa2c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:53.011476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-b1acb7a1-2f6a-4e9e-91f0-52c0b33fcb68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79af81cd177248979f4c73c93a7baa2c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:53.012320 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-b1acb7a1-2f6a-4e9e-91f0-52c0b33fcb68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79af81cd177248979f4c73c93a7baa2c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:53.012994 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-b1acb7a1-2f6a-4e9e-91f0-52c0b33fcb68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79af81cd177248979f4c73c93a7baa2c'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:53.031347 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1384/2800] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:10:52 2022] DELETE /identity/v3/users/79af81cd177248979f4c73c93a7baa2c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:53.036204 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914521d-d5da-4aea-9469-74989dd3b83e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:53.036694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914521d-d5da-4aea-9469-74989dd3b83e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:53.037000 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914521d-d5da-4aea-9469-74989dd3b83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:53.074796 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d914521d-d5da-4aea-9469-74989dd3b83e None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:53.076222 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d914521d-d5da-4aea-9469-74989dd3b83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:53.171174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1417/2801] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:53.180546 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1acb7a1-2f6a-4e9e-91f0-52c0b33fcb68 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:53.183434 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b1acb7a1-2f6a-4e9e-91f0-52c0b33fcb68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:53.285978 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1acb7a1-2f6a-4e9e-91f0-52c0b33fcb68 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:53.295202 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738a6fe-837c-451a-8842-2f72f889a042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:53.295202 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738a6fe-837c-451a-8842-2f72f889a042 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:53.295202 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738a6fe-837c-451a-8842-2f72f889a042 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:53.295202 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738a6fe-837c-451a-8842-2f72f889a042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:53.316629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1738a6fe-837c-451a-8842-2f72f889a042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:53.318793 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1738a6fe-837c-451a-8842-2f72f889a042 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:53.333672 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1385/2802] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:10:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:10:53.736407 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914521d-d5da-4aea-9469-74989dd3b83e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:53.738447 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d914521d-d5da-4aea-9469-74989dd3b83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:53.759851 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914521d-d5da-4aea-9469-74989dd3b83e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:53.765319 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982a101-3ae2-46d3-9ab2-20a5810fda26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:53.766050 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982a101-3ae2-46d3-9ab2-20a5810fda26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:53.766502 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982a101-3ae2-46d3-9ab2-20a5810fda26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:53.766828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982a101-3ae2-46d3-9ab2-20a5810fda26 admin admin] PATH_INFO: `/v3/projects/dea9d1c28eee4b0ca2a1af39ca7d3576` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:53.768447 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1982a101-3ae2-46d3-9ab2-20a5810fda26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dea9d1c28eee4b0ca2a1af39ca7d3576)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:53.770831 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1982a101-3ae2-46d3-9ab2-20a5810fda26 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:53.837632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1982a101-3ae2-46d3-9ab2-20a5810fda26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dea9d1c28eee4b0ca2a1af39ca7d3576', 'request_id': 'req-1982a101-3ae2-46d3-9ab2-20a5810fda26'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:53.844632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1982a101-3ae2-46d3-9ab2-20a5810fda26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dea9d1c28eee4b0ca2a1af39ca7d3576', 'request_id': 'req-1982a101-3ae2-46d3-9ab2-20a5810fda26'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:53.857877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1982a101-3ae2-46d3-9ab2-20a5810fda26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dea9d1c28eee4b0ca2a1af39ca7d3576', 'request_id': 'req-1982a101-3ae2-46d3-9ab2-20a5810fda26'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:53.871814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1982a101-3ae2-46d3-9ab2-20a5810fda26 admin admin] The token cache is being invalidate because project dea9d1c28eee4b0ca2a1af39ca7d3576 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:10:53.872259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1982a101-3ae2-46d3-9ab2-20a5810fda26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:53.872988 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1982a101-3ae2-46d3-9ab2-20a5810fda26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:10:53.879557 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1418/2803] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:10:53 2022] DELETE /identity/v3/projects/dea9d1c28eee4b0ca2a1af39ca7d3576 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:53.881473 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ca254-5b60-4fe1-b102-f3d2aa24a5f6 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:53.881944 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ca254-5b60-4fe1-b102-f3d2aa24a5f6 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:53.882020 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ca254-5b60-4fe1-b102-f3d2aa24a5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:53.906768 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-4c1ca254-5b60-4fe1-b102-f3d2aa24a5f6 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:53.908799 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4c1ca254-5b60-4fe1-b102-f3d2aa24a5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:53.982501 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1386/2804] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:53.984795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba67aa-23c9-4236-be52-129df61597c8 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:53.985197 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba67aa-23c9-4236-be52-129df61597c8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:53.985577 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba67aa-23c9-4236-be52-129df61597c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:54.011321 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-60ba67aa-23c9-4236-be52-129df61597c8 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:54.013194 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-60ba67aa-23c9-4236-be52-129df61597c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:54.022329 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1419/2805] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:54.022773 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ca254-5b60-4fe1-b102-f3d2aa24a5f6 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:54.024817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4c1ca254-5b60-4fe1-b102-f3d2aa24a5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:54.026876 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba67aa-23c9-4236-be52-129df61597c8 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:54.043158 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba67aa-23c9-4236-be52-129df61597c8 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:54.046022 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:54.049531 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f984e29-d54e-43be-b955-b51b2366a0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:54.050158 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f984e29-d54e-43be-b955-b51b2366a0d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:54.050595 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f984e29-d54e-43be-b955-b51b2366a0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:54.050838 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f984e29-d54e-43be-b955-b51b2366a0d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:54.051704 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f984e29-d54e-43be-b955-b51b2366a0d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:10:54.052154 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f984e29-d54e-43be-b955-b51b2366a0d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:54.054139 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f984e29-d54e-43be-b955-b51b2366a0d8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:54.070479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1420/2806] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 13:10:54.074570 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f984e29-d54e-43be-b955-b51b2366a0d8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:54.085362 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f984e29-d54e-43be-b955-b51b2366a0d8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:54.092027 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30125a09-323d-4469-a407-cd9c488e869e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:54.092703 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30125a09-323d-4469-a407-cd9c488e869e admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:54.093540 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30125a09-323d-4469-a407-cd9c488e869e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:54.094191 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30125a09-323d-4469-a407-cd9c488e869e admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:54.095036 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30125a09-323d-4469-a407-cd9c488e869e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:54.097383 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30125a09-323d-4469-a407-cd9c488e869e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:54.148978 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1421/2807] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:10:54 2022] POST /identity/v3/projects => generated 371 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:10:54.149961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ca254-5b60-4fe1-b102-f3d2aa24a5f6 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:54.152342 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30125a09-323d-4469-a407-cd9c488e869e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:54.207773 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:54.239731 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30125a09-323d-4469-a407-cd9c488e869e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:54.245352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dfeb55-5ce7-4c5f-a68e-36b0e6ca887b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:54.246414 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfeb55-5ce7-4c5f-a68e-36b0e6ca887b service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:54.246414 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfeb55-5ce7-4c5f-a68e-36b0e6ca887b service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:54.246813 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfeb55-5ce7-4c5f-a68e-36b0e6ca887b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:54.249464 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f7dfeb55-5ce7-4c5f-a68e-36b0e6ca887b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:54.250345 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf6e4d6-61ce-441a-acd7-eb92c5e58705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:54.251674 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6e4d6-61ce-441a-acd7-eb92c5e58705 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:54.252037 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6e4d6-61ce-441a-acd7-eb92c5e58705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:54.252346 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6e4d6-61ce-441a-acd7-eb92c5e58705 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:54.253193 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf6e4d6-61ce-441a-acd7-eb92c5e58705 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:54.255262 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf6e4d6-61ce-441a-acd7-eb92c5e58705 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:54.265507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dfeb55-5ce7-4c5f-a68e-36b0e6ca887b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:54.267453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dfeb55-5ce7-4c5f-a68e-36b0e6ca887b service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:54.292265 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1422/2808] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:10:54 2022] POST /identity/v3/users => generated 286 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:10:54.293391 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf6e4d6-61ce-441a-acd7-eb92c5e58705 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:54.293530 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1387/2809] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:10:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 13:10:54.308502 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf6e4d6-61ce-441a-acd7-eb92c5e58705 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:54.314263 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30a365-27f7-4c55-bc18-945b091e2a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:54.314939 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30a365-27f7-4c55-bc18-945b091e2a12 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:54.315487 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30a365-27f7-4c55-bc18-945b091e2a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:54.315487 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30a365-27f7-4c55-bc18-945b091e2a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:54.316164 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30a365-27f7-4c55-bc18-945b091e2a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:54.320673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30a365-27f7-4c55-bc18-945b091e2a12 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:54.332513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1423/2810] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:10:54.336537 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dfeb55-5ce7-4c5f-a68e-36b0e6ca887b service nova] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:54.350851 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dfeb55-5ce7-4c5f-a68e-36b0e6ca887b service nova] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:54.355860 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3a36b-bc0e-420f-a797-a8c830257c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:54.357062 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3a36b-bc0e-420f-a797-a8c830257c41 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:54.357336 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3a36b-bc0e-420f-a797-a8c830257c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:54.357748 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3a36b-bc0e-420f-a797-a8c830257c41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:54.358443 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3a36b-bc0e-420f-a797-a8c830257c41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:54.360026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3a36b-bc0e-420f-a797-a8c830257c41 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:54.370295 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1388/2811] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 13:10:54.375039 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30a365-27f7-4c55-bc18-945b091e2a12 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:54.385333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30a365-27f7-4c55-bc18-945b091e2a12 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:54.390679 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85497a05-5755-45b1-9123-929cd9d3dffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:54.391345 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85497a05-5755-45b1-9123-929cd9d3dffc admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:54.391624 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85497a05-5755-45b1-9123-929cd9d3dffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:54.391931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85497a05-5755-45b1-9123-929cd9d3dffc admin admin] PATH_INFO: `/v3/projects/06e0dc4c7d244821bd974b0dea3fc631/users/2dacef0fd30a43d2b29a2eeb67884ec1/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:54.410673 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85497a05-5755-45b1-9123-929cd9d3dffc admin admin] RBAC: Authorizing `identity:create_grant(project_id=06e0dc4c7d244821bd974b0dea3fc631, user_id=2dacef0fd30a43d2b29a2eeb67884ec1, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:54.413371 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85497a05-5755-45b1-9123-929cd9d3dffc admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:54.427683 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1424/2812] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:10:54 2022] PUT /identity/v3/projects/06e0dc4c7d244821bd974b0dea3fc631/users/2dacef0fd30a43d2b29a2eeb67884ec1/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:54.432821 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa156841-fc5b-4e69-ab0a-cb1b2f1a9b08 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:54.433254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa156841-fc5b-4e69-ab0a-cb1b2f1a9b08 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:54.433557 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa156841-fc5b-4e69-ab0a-cb1b2f1a9b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:54.452769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-aa156841-fc5b-4e69-ab0a-cb1b2f1a9b08 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:54.453959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-aa156841-fc5b-4e69-ab0a-cb1b2f1a9b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:54.512733 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1389/2813] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 13:10:54.522745 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85497a05-5755-45b1-9123-929cd9d3dffc admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:54.525093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-85497a05-5755-45b1-9123-929cd9d3dffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:54.582784 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85497a05-5755-45b1-9123-929cd9d3dffc admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:54.586390 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89375c9b-2d22-4f55-8ba0-e9f662047f91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:54.586998 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89375c9b-2d22-4f55-8ba0-e9f662047f91 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:54.587276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89375c9b-2d22-4f55-8ba0-e9f662047f91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:54.587548 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89375c9b-2d22-4f55-8ba0-e9f662047f91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:54.596554 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89375c9b-2d22-4f55-8ba0-e9f662047f91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:54.598227 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89375c9b-2d22-4f55-8ba0-e9f662047f91 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:54.608370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1425/2814] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:10:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:10:58.818245 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7991e-d122-46df-8f1d-b5b4a0b2c019 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:58.818797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7991e-d122-46df-8f1d-b5b4a0b2c019 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:58.818797 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7991e-d122-46df-8f1d-b5b4a0b2c019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:58.853787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-efd7991e-d122-46df-8f1d-b5b4a0b2c019 None None] MFA Rules not processed for user `2dacef0fd30a43d2b29a2eeb67884ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:10:58.855005 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-efd7991e-d122-46df-8f1d-b5b4a0b2c019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:10:58.878695 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:10:58.960658 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1390/2815] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:10:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:10:58.961514 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89375c9b-2d22-4f55-8ba0-e9f662047f91 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:58.971354 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89375c9b-2d22-4f55-8ba0-e9f662047f91 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:58.978147 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c92bed-50bb-4db1-90a1-06c4fd384e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:58.978765 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c92bed-50bb-4db1-90a1-06c4fd384e92 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:58.979041 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c92bed-50bb-4db1-90a1-06c4fd384e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:58.979310 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c92bed-50bb-4db1-90a1-06c4fd384e92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:58.979983 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c92bed-50bb-4db1-90a1-06c4fd384e92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:58.982447 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c92bed-50bb-4db1-90a1-06c4fd384e92 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:59.032690 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1426/2816] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:10:58 2022] POST /identity/v3/projects => generated 373 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:10:59.036504 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7991e-d122-46df-8f1d-b5b4a0b2c019 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:59.090653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7991e-d122-46df-8f1d-b5b4a0b2c019 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:59.182346 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec07d80-4c93-4f43-b16c-71d587bb93c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:59.183222 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec07d80-4c93-4f43-b16c-71d587bb93c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:59.183519 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec07d80-4c93-4f43-b16c-71d587bb93c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:59.183802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec07d80-4c93-4f43-b16c-71d587bb93c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:59.185253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec07d80-4c93-4f43-b16c-71d587bb93c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:59.187385 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec07d80-4c93-4f43-b16c-71d587bb93c5 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:59.220293 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1391/2817] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:10:59 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:10:59.224358 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c92bed-50bb-4db1-90a1-06c4fd384e92 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:59.250240 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c92bed-50bb-4db1-90a1-06c4fd384e92 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:59.255925 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7443e7b-4bfb-4de8-a87b-895d09c1db77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:59.256980 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7443e7b-4bfb-4de8-a87b-895d09c1db77 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:59.257474 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7443e7b-4bfb-4de8-a87b-895d09c1db77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:59.258018 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7443e7b-4bfb-4de8-a87b-895d09c1db77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:59.258785 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7443e7b-4bfb-4de8-a87b-895d09c1db77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:59.260541 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7443e7b-4bfb-4de8-a87b-895d09c1db77 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:59.277671 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1427/2818] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:10:59.279754 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec07d80-4c93-4f43-b16c-71d587bb93c5 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:59.293198 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec07d80-4c93-4f43-b16c-71d587bb93c5 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:59.298706 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f5913c-f796-49cf-8b9e-2f71743bcd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:59.299461 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5913c-f796-49cf-8b9e-2f71743bcd21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:59.299808 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5913c-f796-49cf-8b9e-2f71743bcd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:59.300155 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5913c-f796-49cf-8b9e-2f71743bcd21 admin admin] PATH_INFO: `/v3/projects/d7cfbcb88aa34ad1bb98a5dbfd849cb3/users/2733d80bc47b432abf7adede6ef1cbb0/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:59.321249 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f5913c-f796-49cf-8b9e-2f71743bcd21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7cfbcb88aa34ad1bb98a5dbfd849cb3, user_id=2733d80bc47b432abf7adede6ef1cbb0, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:59.323941 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f5913c-f796-49cf-8b9e-2f71743bcd21 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:59.339313 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1392/2819] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:10:59 2022] PUT /identity/v3/projects/d7cfbcb88aa34ad1bb98a5dbfd849cb3/users/2733d80bc47b432abf7adede6ef1cbb0/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:10:59.345400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7443e7b-4bfb-4de8-a87b-895d09c1db77 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:59.357704 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7443e7b-4bfb-4de8-a87b-895d09c1db77 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:59.424859 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c47fc6-7161-4147-ad98-a5e2ae235fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:59.425488 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c47fc6-7161-4147-ad98-a5e2ae235fcc admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:59.425767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c47fc6-7161-4147-ad98-a5e2ae235fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:59.426036 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c47fc6-7161-4147-ad98-a5e2ae235fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:59.426726 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c47fc6-7161-4147-ad98-a5e2ae235fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:59.428526 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c47fc6-7161-4147-ad98-a5e2ae235fcc admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:59.441461 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1428/2820] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:10:59.445404 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f5913c-f796-49cf-8b9e-2f71743bcd21 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:10:59.456626 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f5913c-f796-49cf-8b9e-2f71743bcd21 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:10:59.461975 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24fac16-16af-4690-a4d5-c9005e008bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:10:59.462669 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24fac16-16af-4690-a4d5-c9005e008bb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:10:59.462949 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24fac16-16af-4690-a4d5-c9005e008bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:10:59.463255 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24fac16-16af-4690-a4d5-c9005e008bb5 admin admin] PATH_INFO: `/v3/projects/d7cfbcb88aa34ad1bb98a5dbfd849cb3/users/2733d80bc47b432abf7adede6ef1cbb0/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:10:59.465374 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24fac16-16af-4690-a4d5-c9005e008bb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7cfbcb88aa34ad1bb98a5dbfd849cb3, user_id=2733d80bc47b432abf7adede6ef1cbb0, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:10:59.468583 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24fac16-16af-4690-a4d5-c9005e008bb5 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:10:59.485295 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1393/2821] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:10:59 2022] PUT /identity/v3/projects/d7cfbcb88aa34ad1bb98a5dbfd849cb3/users/2733d80bc47b432abf7adede6ef1cbb0/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:11:02.294941 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1eb64-a3bc-4a72-88a6-85d44a38ec02 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:11:02.295744 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1eb64-a3bc-4a72-88a6-85d44a38ec02 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:11:02.296193 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1eb64-a3bc-4a72-88a6-85d44a38ec02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:11:02.309049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:11:02.332784 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d8b1eb64-a3bc-4a72-88a6-85d44a38ec02 None None] MFA Rules not processed for user `4f71cfae23f74c96b1cf3329804f369e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:11:02.334515 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d8b1eb64-a3bc-4a72-88a6-85d44a38ec02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:11:02.461656 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1429/2822] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:11:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:11:03.004635 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30cd578-135b-4f71-a90a-1f2709738380 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:11:03.005088 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30cd578-135b-4f71-a90a-1f2709738380 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:11:03.005206 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30cd578-135b-4f71-a90a-1f2709738380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:11:03.052075 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-c30cd578-135b-4f71-a90a-1f2709738380 None None] MFA Rules not processed for user `c810b52785b94c0cab4fd9f0d6e65825`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:11:03.053809 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c30cd578-135b-4f71-a90a-1f2709738380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:11:03.067433 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1eb64-a3bc-4a72-88a6-85d44a38ec02 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:11:03.068910 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d8b1eb64-a3bc-4a72-88a6-85d44a38ec02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:11:03.178096 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1394/2823] 10.222.0.20 () {58 vars in 967 bytes} [Sat Aug 6 13:11:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:11:03.215887 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1eb64-a3bc-4a72-88a6-85d44a38ec02 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:11:03.232691 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8beed2b-a1c2-449d-98a9-1410b456ecec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:11:03.233431 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8beed2b-a1c2-449d-98a9-1410b456ecec service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:11:03.233772 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8beed2b-a1c2-449d-98a9-1410b456ecec service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:11:03.234117 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8beed2b-a1c2-449d-98a9-1410b456ecec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:11:03.237833 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e8beed2b-a1c2-449d-98a9-1410b456ecec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:11:03.250487 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30cd578-135b-4f71-a90a-1f2709738380 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:11:03.252033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c30cd578-135b-4f71-a90a-1f2709738380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:11:03.274418 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8beed2b-a1c2-449d-98a9-1410b456ecec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:11:03.277141 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8beed2b-a1c2-449d-98a9-1410b456ecec service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:11:03.288824 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:11:03.295632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1430/2824] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:11:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 13:11:03.453660 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30cd578-135b-4f71-a90a-1f2709738380 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:11:03.464849 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21548e-7a52-4867-9403-a9fbd4da38e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efc792a2655439f8b8070010059b6bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:11:03.465686 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21548e-7a52-4867-9403-a9fbd4da38e6 service swift] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:11:03.466065 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21548e-7a52-4867-9403-a9fbd4da38e6 service swift] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:11:03.466442 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21548e-7a52-4867-9403-a9fbd4da38e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:11:03.480543 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21548e-7a52-4867-9403-a9fbd4da38e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:11:03.483031 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21548e-7a52-4867-9403-a9fbd4da38e6 service swift] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:11:03.511253 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1395/2825] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Aug 6 13:11:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 06 13:11:05.715761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c041de5-d233-440d-b06d-8e344670fa1c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:11:05.716280 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c041de5-d233-440d-b06d-8e344670fa1c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:11:05.716280 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c041de5-d233-440d-b06d-8e344670fa1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:11:05.785036 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2c041de5-d233-440d-b06d-8e344670fa1c None None] MFA Rules not processed for user `2733d80bc47b432abf7adede6ef1cbb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:11:05.786276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2c041de5-d233-440d-b06d-8e344670fa1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:11:05.896806 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1431/2826] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:11:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:11:05.910354 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21548e-7a52-4867-9403-a9fbd4da38e6 service swift] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:11:05.914163 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-fb21548e-7a52-4867-9403-a9fbd4da38e6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:11:06.068045 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21548e-7a52-4867-9403-a9fbd4da38e6 service swift] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:11:06.074492 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60ac88-4953-4c82-ad68-4e35eb217c4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:11:06.075477 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60ac88-4953-4c82-ad68-4e35eb217c4d service nova] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:11:06.075933 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60ac88-4953-4c82-ad68-4e35eb217c4d service nova] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:11:06.076415 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60ac88-4953-4c82-ad68-4e35eb217c4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:11:06.189190 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a60ac88-4953-4c82-ad68-4e35eb217c4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:11:06.191021 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a60ac88-4953-4c82-ad68-4e35eb217c4d service nova] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:11:06.208979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1396/2827] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:11:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:11:08.567174 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e0e02-501e-4cd9-9b05-498912dae55f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:11:08.567646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e0e02-501e-4cd9-9b05-498912dae55f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:11:08.567752 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e0e02-501e-4cd9-9b05-498912dae55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:11:08.599331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-635e0e02-501e-4cd9-9b05-498912dae55f None None] MFA Rules not processed for user `2dacef0fd30a43d2b29a2eeb67884ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:11:08.600870 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-635e0e02-501e-4cd9-9b05-498912dae55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:11:08.610482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1432/2828] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:11:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:11:12.971764 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60ac88-4953-4c82-ad68-4e35eb217c4d service nova] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:11:12.973523 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6a60ac88-4953-4c82-ad68-4e35eb217c4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:11:13.033978 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:11:13.055917 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60ac88-4953-4c82-ad68-4e35eb217c4d service nova] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:11:13.060883 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b602c-171b-43fd-ab96-f2f14aa1339a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:11:13.061514 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b602c-171b-43fd-ab96-f2f14aa1339a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:11:13.061794 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b602c-171b-43fd-ab96-f2f14aa1339a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:11:13.062077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b602c-171b-43fd-ab96-f2f14aa1339a admin admin] PATH_INFO: `/v3/users/3188163f95d04235b5de120354ea4997` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:11:13.063204 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b602c-171b-43fd-ab96-f2f14aa1339a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3188163f95d04235b5de120354ea4997)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:11:13.065139 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b602c-171b-43fd-ab96-f2f14aa1339a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:11:13.111007 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-447b602c-171b-43fd-ab96-f2f14aa1339a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3188163f95d04235b5de120354ea4997'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:13.117919 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-447b602c-171b-43fd-ab96-f2f14aa1339a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3188163f95d04235b5de120354ea4997'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:13.138750 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-447b602c-171b-43fd-ab96-f2f14aa1339a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3188163f95d04235b5de120354ea4997'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:13.139469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-447b602c-171b-43fd-ab96-f2f14aa1339a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3188163f95d04235b5de120354ea4997'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:13.140067 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-447b602c-171b-43fd-ab96-f2f14aa1339a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3188163f95d04235b5de120354ea4997'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:13.153228 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1397/2829] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:11:12 2022] DELETE /identity/v3/users/3188163f95d04235b5de120354ea4997 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:11:13.334222 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e0e02-501e-4cd9-9b05-498912dae55f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:11:13.348179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e0e02-501e-4cd9-9b05-498912dae55f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:11:13.378879 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:11:13.409609 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ed99b-b198-47f3-aa0f-43015840c18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:11:13.410251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ed99b-b198-47f3-aa0f-43015840c18e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:11:13.410538 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ed99b-b198-47f3-aa0f-43015840c18e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:11:13.410813 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ed99b-b198-47f3-aa0f-43015840c18e admin admin] PATH_INFO: `/v3/projects/f7477fc540f843b4936879b9425896f3` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:11:13.412001 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842ed99b-b198-47f3-aa0f-43015840c18e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7477fc540f843b4936879b9425896f3)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:11:13.413961 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842ed99b-b198-47f3-aa0f-43015840c18e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:11:13.466491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-842ed99b-b198-47f3-aa0f-43015840c18e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7477fc540f843b4936879b9425896f3', 'request_id': 'req-842ed99b-b198-47f3-aa0f-43015840c18e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:13.473010 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-842ed99b-b198-47f3-aa0f-43015840c18e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7477fc540f843b4936879b9425896f3', 'request_id': 'req-842ed99b-b198-47f3-aa0f-43015840c18e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:13.482100 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-842ed99b-b198-47f3-aa0f-43015840c18e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7477fc540f843b4936879b9425896f3', 'request_id': 'req-842ed99b-b198-47f3-aa0f-43015840c18e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:13.493931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-842ed99b-b198-47f3-aa0f-43015840c18e admin admin] The token cache is being invalidate because project f7477fc540f843b4936879b9425896f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:11:13.494289 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-842ed99b-b198-47f3-aa0f-43015840c18e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:13.494925 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-842ed99b-b198-47f3-aa0f-43015840c18e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:13.497162 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1433/2830] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:11:13 2022] DELETE /identity/v3/projects/f7477fc540f843b4936879b9425896f3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:11:13.501248 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b602c-171b-43fd-ab96-f2f14aa1339a admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:11:13.502708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-447b602c-171b-43fd-ab96-f2f14aa1339a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:11:13.515972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b602c-171b-43fd-ab96-f2f14aa1339a admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:11:13.594719 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3190591-f938-4504-a91d-0bc2d135c341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:11:13.595618 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3190591-f938-4504-a91d-0bc2d135c341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:11:13.595994 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3190591-f938-4504-a91d-0bc2d135c341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:11:13.596420 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3190591-f938-4504-a91d-0bc2d135c341 admin admin] PATH_INFO: `/v3/users/0b67f99b002c4879b006efb048c5aaf4` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:11:13.598054 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3190591-f938-4504-a91d-0bc2d135c341 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b67f99b002c4879b006efb048c5aaf4)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:11:13.600285 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3190591-f938-4504-a91d-0bc2d135c341 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:11:13.658416 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c3190591-f938-4504-a91d-0bc2d135c341 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b67f99b002c4879b006efb048c5aaf4'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:13.665325 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c3190591-f938-4504-a91d-0bc2d135c341 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b67f99b002c4879b006efb048c5aaf4'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:13.671281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:11:13.683443 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c3190591-f938-4504-a91d-0bc2d135c341 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b67f99b002c4879b006efb048c5aaf4'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:13.684613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c3190591-f938-4504-a91d-0bc2d135c341 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b67f99b002c4879b006efb048c5aaf4'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:13.685276 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-c3190591-f938-4504-a91d-0bc2d135c341 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b67f99b002c4879b006efb048c5aaf4'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:13.700473 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1398/2831] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:11:13 2022] DELETE /identity/v3/users/0b67f99b002c4879b006efb048c5aaf4 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:11:13.923479 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ed99b-b198-47f3-aa0f-43015840c18e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:11:13.925714 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-842ed99b-b198-47f3-aa0f-43015840c18e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:11:13.948348 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ed99b-b198-47f3-aa0f-43015840c18e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:11:14.026611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9495ce-57d7-4c65-b768-ed5df802f983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:11:14.028063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9495ce-57d7-4c65-b768-ed5df802f983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:11:14.028610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9495ce-57d7-4c65-b768-ed5df802f983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:11:14.028984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9495ce-57d7-4c65-b768-ed5df802f983 admin admin] PATH_INFO: `/v3/projects/257ddc6f37a64b71b6eb8ce15fc3e79f` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:11:14.030392 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9495ce-57d7-4c65-b768-ed5df802f983 admin admin] RBAC: Authorizing `identity:delete_project(project_id=257ddc6f37a64b71b6eb8ce15fc3e79f)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:11:14.032675 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9495ce-57d7-4c65-b768-ed5df802f983 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:11:14.100148 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8f9495ce-57d7-4c65-b768-ed5df802f983 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '257ddc6f37a64b71b6eb8ce15fc3e79f', 'request_id': 'req-8f9495ce-57d7-4c65-b768-ed5df802f983'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:14.111374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8f9495ce-57d7-4c65-b768-ed5df802f983 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '257ddc6f37a64b71b6eb8ce15fc3e79f', 'request_id': 'req-8f9495ce-57d7-4c65-b768-ed5df802f983'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:14.114256 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:11:14.123048 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8f9495ce-57d7-4c65-b768-ed5df802f983 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '257ddc6f37a64b71b6eb8ce15fc3e79f', 'request_id': 'req-8f9495ce-57d7-4c65-b768-ed5df802f983'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:14.137871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8f9495ce-57d7-4c65-b768-ed5df802f983 admin admin] The token cache is being invalidate because project 257ddc6f37a64b71b6eb8ce15fc3e79f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:11:14.138303 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8f9495ce-57d7-4c65-b768-ed5df802f983 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:14.139028 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8f9495ce-57d7-4c65-b768-ed5df802f983 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:14.140634 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1434/2832] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:11:13 2022] DELETE /identity/v3/projects/257ddc6f37a64b71b6eb8ce15fc3e79f => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:11:23.695823 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=6,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:11:24.135484 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=1,keystone:SELECT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:11:25.160987 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3190591-f938-4504-a91d-0bc2d135c341 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:11:25.162427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-c3190591-f938-4504-a91d-0bc2d135c341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:11:25.217996 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3190591-f938-4504-a91d-0bc2d135c341 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:11:25.266261 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a41ed7-5ed7-478d-8ac3-90133b86c710 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:11:25.266261 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a41ed7-5ed7-478d-8ac3-90133b86c710 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:11:25.266261 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a41ed7-5ed7-478d-8ac3-90133b86c710 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:11:25.266261 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a41ed7-5ed7-478d-8ac3-90133b86c710 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:11:25.267629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-98a41ed7-5ed7-478d-8ac3-90133b86c710 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:11:25.324882 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a41ed7-5ed7-478d-8ac3-90133b86c710 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:11:25.326308 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a41ed7-5ed7-478d-8ac3-90133b86c710 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:11:25.335196 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1399/2833] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:11:25 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:11:33.707501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afeba2-993a-4d03-849a-3948f4e67432 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:11:33.707969 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afeba2-993a-4d03-849a-3948f4e67432 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:11:33.708057 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afeba2-993a-4d03-849a-3948f4e67432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:11:33.727778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c1afeba2-993a-4d03-849a-3948f4e67432 None None] MFA Rules not processed for user `c810b52785b94c0cab4fd9f0d6e65825`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:11:33.729018 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c1afeba2-993a-4d03-849a-3948f4e67432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:11:33.794650 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1435/2834] 10.222.0.20 () {58 vars in 1009 bytes} [Sat Aug 6 13:11:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:11:33.806695 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a41ed7-5ed7-478d-8ac3-90133b86c710 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:11:33.808070 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-98a41ed7-5ed7-478d-8ac3-90133b86c710 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:11:33.879556 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a41ed7-5ed7-478d-8ac3-90133b86c710 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:11:33.884655 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b437ae0-caed-42ed-949f-cb057b5bd54f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efc792a2655439f8b8070010059b6bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:11:33.885295 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b437ae0-caed-42ed-949f-cb057b5bd54f service swift] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:11:33.885572 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b437ae0-caed-42ed-949f-cb057b5bd54f service swift] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:11:33.885843 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b437ae0-caed-42ed-949f-cb057b5bd54f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:11:33.894490 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b437ae0-caed-42ed-949f-cb057b5bd54f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:11:33.896109 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b437ae0-caed-42ed-949f-cb057b5bd54f service swift] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:11:33.900960 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=100 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:11:33.904979 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1400/2835] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Aug 6 13:11:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 13:11:40.852847 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afeba2-993a-4d03-849a-3948f4e67432 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:11:40.854994 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c1afeba2-993a-4d03-849a-3948f4e67432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:11:41.006270 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afeba2-993a-4d03-849a-3948f4e67432 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:11:41.018645 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f104ad-0af9-4f34-ab01-16938fa92612 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:11:41.019766 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f104ad-0af9-4f34-ab01-16938fa92612 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:11:41.019986 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f104ad-0af9-4f34-ab01-16938fa92612 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:11:41.026696 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f104ad-0af9-4f34-ab01-16938fa92612 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:11:41.029043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-23f104ad-0af9-4f34-ab01-16938fa92612 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:11:41.050946 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f104ad-0af9-4f34-ab01-16938fa92612 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:11:41.053280 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f104ad-0af9-4f34-ab01-16938fa92612 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:11:41.077877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1436/2836] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:11:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:11:49.177865 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25052a-92b7-4add-8c59-1fba6183bf3d None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:11:49.177865 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25052a-92b7-4add-8c59-1fba6183bf3d None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:11:49.178495 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25052a-92b7-4add-8c59-1fba6183bf3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:11:49.197540 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-5a25052a-92b7-4add-8c59-1fba6183bf3d None None] MFA Rules not processed for user `c810b52785b94c0cab4fd9f0d6e65825`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:11:49.198737 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5a25052a-92b7-4add-8c59-1fba6183bf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:11:49.206285 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1401/2837] 10.222.0.20 () {58 vars in 967 bytes} [Sat Aug 6 13:11:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:11:49.218229 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f104ad-0af9-4f34-ab01-16938fa92612 service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:11:49.230702 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f104ad-0af9-4f34-ab01-16938fa92612 service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:11:49.236408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe7495-6187-42e1-9581-f6f9ff986bcd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2efc792a2655439f8b8070010059b6bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:11:49.237128 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe7495-6187-42e1-9581-f6f9ff986bcd service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:11:49.237408 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe7495-6187-42e1-9581-f6f9ff986bcd service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:11:49.237804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe7495-6187-42e1-9581-f6f9ff986bcd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:11:49.247522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe7495-6187-42e1-9581-f6f9ff986bcd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:11:49.249854 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe7495-6187-42e1-9581-f6f9ff986bcd service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:11:49.258643 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1437/2838] 10.222.0.20 () {60 vars in 1384 bytes} [Sat Aug 6 13:11:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 06 13:11:51.178180 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a25052a-92b7-4add-8c59-1fba6183bf3d None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:11:51.179717 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5a25052a-92b7-4add-8c59-1fba6183bf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:11:51.213292 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe7495-6187-42e1-9581-f6f9ff986bcd service swift] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:11:51.214939 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bebe7495-6187-42e1-9581-f6f9ff986bcd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:11:51.235671 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe7495-6187-42e1-9581-f6f9ff986bcd service swift] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:11:51.241594 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de23a5-61f2-45b1-8588-fc0f92ba69f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '524b728bf63c4ca88e0f25e531cf5056', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:11:51.242220 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de23a5-61f2-45b1-8588-fc0f92ba69f7 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:11:51.242501 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de23a5-61f2-45b1-8588-fc0f92ba69f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:11:51.242985 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de23a5-61f2-45b1-8588-fc0f92ba69f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:11:51.244869 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-29de23a5-61f2-45b1-8588-fc0f92ba69f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:11:51.257819 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a25052a-92b7-4add-8c59-1fba6183bf3d None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:11:51.260900 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de23a5-61f2-45b1-8588-fc0f92ba69f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:11:51.262508 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de23a5-61f2-45b1-8588-fc0f92ba69f7 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:11:51.265751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eca4ac-d6be-4360-8436-49d2973c8a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:11:51.266409 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eca4ac-d6be-4360-8436-49d2973c8a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:11:51.266692 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eca4ac-d6be-4360-8436-49d2973c8a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:11:51.266987 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eca4ac-d6be-4360-8436-49d2973c8a4e admin admin] PATH_INFO: `/v3/users/4f71cfae23f74c96b1cf3329804f369e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:11:51.268212 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eca4ac-d6be-4360-8436-49d2973c8a4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f71cfae23f74c96b1cf3329804f369e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:11:51.271085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eca4ac-d6be-4360-8436-49d2973c8a4e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:11:51.275617 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1438/2839] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:11:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 13:11:51.322753 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-43eca4ac-d6be-4360-8436-49d2973c8a4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f71cfae23f74c96b1cf3329804f369e'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:51.330060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-43eca4ac-d6be-4360-8436-49d2973c8a4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f71cfae23f74c96b1cf3329804f369e'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:51.342222 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-43eca4ac-d6be-4360-8436-49d2973c8a4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f71cfae23f74c96b1cf3329804f369e'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:51.342883 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-43eca4ac-d6be-4360-8436-49d2973c8a4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f71cfae23f74c96b1cf3329804f369e'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:51.343500 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-43eca4ac-d6be-4360-8436-49d2973c8a4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f71cfae23f74c96b1cf3329804f369e'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:51.358071 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1402/2840] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:11:51 2022] DELETE /identity/v3/users/4f71cfae23f74c96b1cf3329804f369e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:11:51.547482 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de23a5-61f2-45b1-8588-fc0f92ba69f7 service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:11:51.558638 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de23a5-61f2-45b1-8588-fc0f92ba69f7 service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:11:51.613816 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:11:51.626732 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69de7a8-0a4e-4e12-9d35-dc950b18debf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:11:51.627510 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69de7a8-0a4e-4e12-9d35-dc950b18debf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:11:51.627795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69de7a8-0a4e-4e12-9d35-dc950b18debf admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:11:51.628068 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69de7a8-0a4e-4e12-9d35-dc950b18debf admin admin] PATH_INFO: `/v3/projects/b8ba2e48caca474f860946df0c9d0baa` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:11:51.629262 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69de7a8-0a4e-4e12-9d35-dc950b18debf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8ba2e48caca474f860946df0c9d0baa)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:11:51.631172 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69de7a8-0a4e-4e12-9d35-dc950b18debf admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:11:51.681248 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b69de7a8-0a4e-4e12-9d35-dc950b18debf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8ba2e48caca474f860946df0c9d0baa', 'request_id': 'req-b69de7a8-0a4e-4e12-9d35-dc950b18debf'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:51.686600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b69de7a8-0a4e-4e12-9d35-dc950b18debf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8ba2e48caca474f860946df0c9d0baa', 'request_id': 'req-b69de7a8-0a4e-4e12-9d35-dc950b18debf'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:51.693869 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b69de7a8-0a4e-4e12-9d35-dc950b18debf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8ba2e48caca474f860946df0c9d0baa', 'request_id': 'req-b69de7a8-0a4e-4e12-9d35-dc950b18debf'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:51.705490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b69de7a8-0a4e-4e12-9d35-dc950b18debf admin admin] The token cache is being invalidate because project b8ba2e48caca474f860946df0c9d0baa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:11:51.705783 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b69de7a8-0a4e-4e12-9d35-dc950b18debf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:51.706409 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b69de7a8-0a4e-4e12-9d35-dc950b18debf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:11:51.707942 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1439/2841] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:11:51 2022] DELETE /identity/v3/projects/b8ba2e48caca474f860946df0c9d0baa => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:11:54.480417 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eca4ac-d6be-4360-8436-49d2973c8a4e admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:11:54.482125 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-43eca4ac-d6be-4360-8436-49d2973c8a4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:11:54.537212 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eca4ac-d6be-4360-8436-49d2973c8a4e admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:11:54.561918 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:11:54.586503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a666682-eaac-425e-9ac5-e9f407e29a22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:11:54.587113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a666682-eaac-425e-9ac5-e9f407e29a22 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:11:54.587392 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a666682-eaac-425e-9ac5-e9f407e29a22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:11:54.587667 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a666682-eaac-425e-9ac5-e9f407e29a22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:11:54.589443 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5a666682-eaac-425e-9ac5-e9f407e29a22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:11:54.649782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a666682-eaac-425e-9ac5-e9f407e29a22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:11:54.651273 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a666682-eaac-425e-9ac5-e9f407e29a22 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:11:54.662585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1403/2842] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:11:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:12:01.702352 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=29,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:12:04.656984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=37 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:12:30.651125 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69de7a8-0a4e-4e12-9d35-dc950b18debf admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:12:30.652607 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b69de7a8-0a4e-4e12-9d35-dc950b18debf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:12:30.684903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69de7a8-0a4e-4e12-9d35-dc950b18debf admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:12:30.688518 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c787e95-0bad-4286-a14e-12c29d4a0a70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:12:30.689120 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c787e95-0bad-4286-a14e-12c29d4a0a70 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:12:30.689400 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c787e95-0bad-4286-a14e-12c29d4a0a70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:12:30.689672 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c787e95-0bad-4286-a14e-12c29d4a0a70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:12:30.691388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3c787e95-0bad-4286-a14e-12c29d4a0a70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:12:30.754447 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c787e95-0bad-4286-a14e-12c29d4a0a70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:12:30.755883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c787e95-0bad-4286-a14e-12c29d4a0a70 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:12:30.765698 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1440/2843] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:12:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:12:33.856396 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a666682-eaac-425e-9ac5-e9f407e29a22 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:12:33.858613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5a666682-eaac-425e-9ac5-e9f407e29a22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:12:33.885223 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a666682-eaac-425e-9ac5-e9f407e29a22 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:12:33.889965 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109b072-e3d7-47c4-bacc-a082b1b046b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:12:33.890669 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109b072-e3d7-47c4-bacc-a082b1b046b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:12:33.890951 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109b072-e3d7-47c4-bacc-a082b1b046b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:12:33.891222 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109b072-e3d7-47c4-bacc-a082b1b046b6 admin admin] PATH_INFO: `/v3/users/7c520a94e20245ca9cc898dee3ab89bc` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:12:33.892427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109b072-e3d7-47c4-bacc-a082b1b046b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c520a94e20245ca9cc898dee3ab89bc)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:12:33.894331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109b072-e3d7-47c4-bacc-a082b1b046b6 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:12:33.944906 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e109b072-e3d7-47c4-bacc-a082b1b046b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c520a94e20245ca9cc898dee3ab89bc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:12:33.956327 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e109b072-e3d7-47c4-bacc-a082b1b046b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c520a94e20245ca9cc898dee3ab89bc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:12:33.972344 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e109b072-e3d7-47c4-bacc-a082b1b046b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c520a94e20245ca9cc898dee3ab89bc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:12:33.972975 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e109b072-e3d7-47c4-bacc-a082b1b046b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c520a94e20245ca9cc898dee3ab89bc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:12:33.973575 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-e109b072-e3d7-47c4-bacc-a082b1b046b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c520a94e20245ca9cc898dee3ab89bc'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:12:33.990070 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1404/2844] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:12:33 2022] DELETE /identity/v3/users/7c520a94e20245ca9cc898dee3ab89bc => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:12:34.169778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c787e95-0bad-4286-a14e-12c29d4a0a70 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:12:34.179858 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c787e95-0bad-4286-a14e-12c29d4a0a70 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:12:34.246199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9df12-4411-4086-9779-94aeccf8b31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:12:34.246836 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9df12-4411-4086-9779-94aeccf8b31a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:12:34.247117 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9df12-4411-4086-9779-94aeccf8b31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:12:34.247393 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9df12-4411-4086-9779-94aeccf8b31a admin admin] PATH_INFO: `/v3/projects/6dea90217d154c2ab843c86c2a5ebb49` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:12:34.257632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9df12-4411-4086-9779-94aeccf8b31a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dea90217d154c2ab843c86c2a5ebb49)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:12:34.259522 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9df12-4411-4086-9779-94aeccf8b31a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:12:34.305752 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-52b9df12-4411-4086-9779-94aeccf8b31a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dea90217d154c2ab843c86c2a5ebb49', 'request_id': 'req-52b9df12-4411-4086-9779-94aeccf8b31a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:12:34.311333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-52b9df12-4411-4086-9779-94aeccf8b31a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dea90217d154c2ab843c86c2a5ebb49', 'request_id': 'req-52b9df12-4411-4086-9779-94aeccf8b31a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:12:34.318618 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-52b9df12-4411-4086-9779-94aeccf8b31a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dea90217d154c2ab843c86c2a5ebb49', 'request_id': 'req-52b9df12-4411-4086-9779-94aeccf8b31a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:12:34.328423 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:12:34.335323 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-52b9df12-4411-4086-9779-94aeccf8b31a admin admin] The token cache is being invalidate because project 6dea90217d154c2ab843c86c2a5ebb49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:12:34.335912 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-52b9df12-4411-4086-9779-94aeccf8b31a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:12:34.336889 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-52b9df12-4411-4086-9779-94aeccf8b31a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:12:34.338471 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1441/2845] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:12:34 2022] DELETE /identity/v3/projects/6dea90217d154c2ab843c86c2a5ebb49 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:12:34.345850 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e446d1b-acbd-41bc-9c4e-da33ca94a0fd None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:12:34.346141 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e446d1b-acbd-41bc-9c4e-da33ca94a0fd None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:12:34.346681 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e446d1b-acbd-41bc-9c4e-da33ca94a0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:12:34.367003 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-7e446d1b-acbd-41bc-9c4e-da33ca94a0fd None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:12:34.368371 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7e446d1b-acbd-41bc-9c4e-da33ca94a0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:12:34.426106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1405/2846] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:12:34.434778 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e735e-afd8-45ff-abe3-ea4eaea33198 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:12:34.435065 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e735e-afd8-45ff-abe3-ea4eaea33198 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:12:34.435351 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e735e-afd8-45ff-abe3-ea4eaea33198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:12:34.454873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-295e735e-afd8-45ff-abe3-ea4eaea33198 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:12:34.456066 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-295e735e-afd8-45ff-abe3-ea4eaea33198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:12:34.463873 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1442/2847] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:12:34.468087 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e446d1b-acbd-41bc-9c4e-da33ca94a0fd None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:12:34.477908 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e446d1b-acbd-41bc-9c4e-da33ca94a0fd None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:12:34.482476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2caa29-8d4e-4eac-af0a-9c740312bb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:12:34.483069 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2caa29-8d4e-4eac-af0a-9c740312bb56 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:12:34.483347 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2caa29-8d4e-4eac-af0a-9c740312bb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:12:34.483617 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2caa29-8d4e-4eac-af0a-9c740312bb56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:12:34.484319 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2caa29-8d4e-4eac-af0a-9c740312bb56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:12:34.484632 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2caa29-8d4e-4eac-af0a-9c740312bb56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:12:34.486205 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2caa29-8d4e-4eac-af0a-9c740312bb56 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:12:34.497846 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1406/2848] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 13:12:34.501919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e735e-afd8-45ff-abe3-ea4eaea33198 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:12:34.510994 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e735e-afd8-45ff-abe3-ea4eaea33198 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:12:34.516213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bbe7e-fd83-42cd-b042-81cfcecbb00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:12:34.516840 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bbe7e-fd83-42cd-b042-81cfcecbb00c admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:12:34.517117 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bbe7e-fd83-42cd-b042-81cfcecbb00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:12:34.517387 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bbe7e-fd83-42cd-b042-81cfcecbb00c admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:12:34.518037 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793bbe7e-fd83-42cd-b042-81cfcecbb00c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:12:34.519903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793bbe7e-fd83-42cd-b042-81cfcecbb00c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:12:34.545440 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1443/2849] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:12:34 2022] POST /identity/v3/projects => generated 377 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:12:34.549421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2caa29-8d4e-4eac-af0a-9c740312bb56 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:12:34.559667 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2caa29-8d4e-4eac-af0a-9c740312bb56 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:12:34.605444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:12:34.616465 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72c0f9-6213-4bc5-8565-0d5c42ad97f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:12:34.617069 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72c0f9-6213-4bc5-8565-0d5c42ad97f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:12:34.617346 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72c0f9-6213-4bc5-8565-0d5c42ad97f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:12:34.617618 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72c0f9-6213-4bc5-8565-0d5c42ad97f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:12:34.618287 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72c0f9-6213-4bc5-8565-0d5c42ad97f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:12:34.620255 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72c0f9-6213-4bc5-8565-0d5c42ad97f3 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:12:34.641424 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1407/2850] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:12:34 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:12:34.645464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bbe7e-fd83-42cd-b042-81cfcecbb00c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:12:34.655895 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bbe7e-fd83-42cd-b042-81cfcecbb00c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:12:34.660600 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2598e43-7773-4ae8-b5a3-2e259b0ae87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:12:34.661204 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2598e43-7773-4ae8-b5a3-2e259b0ae87f admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:12:34.661488 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2598e43-7773-4ae8-b5a3-2e259b0ae87f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:12:34.661764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2598e43-7773-4ae8-b5a3-2e259b0ae87f admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:12:34.662466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2598e43-7773-4ae8-b5a3-2e259b0ae87f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:12:34.664038 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2598e43-7773-4ae8-b5a3-2e259b0ae87f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:12:34.674180 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1444/2851] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:12:34.677988 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72c0f9-6213-4bc5-8565-0d5c42ad97f3 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:12:34.688113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72c0f9-6213-4bc5-8565-0d5c42ad97f3 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:12:34.692708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28dade-3ef0-40e1-9fb3-4d2ff499a6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:12:34.693294 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28dade-3ef0-40e1-9fb3-4d2ff499a6f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:12:34.693574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28dade-3ef0-40e1-9fb3-4d2ff499a6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:12:34.693846 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28dade-3ef0-40e1-9fb3-4d2ff499a6f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:12:34.694545 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28dade-3ef0-40e1-9fb3-4d2ff499a6f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:12:34.696130 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28dade-3ef0-40e1-9fb3-4d2ff499a6f9 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:12:34.706959 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1408/2852] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:12:34.710806 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2598e43-7773-4ae8-b5a3-2e259b0ae87f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:12:34.720539 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2598e43-7773-4ae8-b5a3-2e259b0ae87f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:12:34.725383 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c308c-0f50-422c-abd0-4914ef32f4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:12:34.726022 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c308c-0f50-422c-abd0-4914ef32f4aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:12:34.726304 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c308c-0f50-422c-abd0-4914ef32f4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:12:34.726581 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c308c-0f50-422c-abd0-4914ef32f4aa admin admin] PATH_INFO: `/v3/projects/8eaf312d87f649d5a9d63909d248473d/users/8b021907238445e1866c7dfa25894deb/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:12:34.741377 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287c308c-0f50-422c-abd0-4914ef32f4aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eaf312d87f649d5a9d63909d248473d, user_id=8b021907238445e1866c7dfa25894deb, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:12:34.744015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287c308c-0f50-422c-abd0-4914ef32f4aa admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:12:34.757145 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1445/2853] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:12:34 2022] PUT /identity/v3/projects/8eaf312d87f649d5a9d63909d248473d/users/8b021907238445e1866c7dfa25894deb/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:12:34.769023 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9680d70-7f34-41a2-a01b-d4edd701b963 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:12:34.769321 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9680d70-7f34-41a2-a01b-d4edd701b963 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:12:34.769587 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9680d70-7f34-41a2-a01b-d4edd701b963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:12:34.797068 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-a9680d70-7f34-41a2-a01b-d4edd701b963 None None] MFA Rules not processed for user `8b021907238445e1866c7dfa25894deb`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:12:34.798258 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-a9680d70-7f34-41a2-a01b-d4edd701b963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:12:34.860732 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1409/2854] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:12:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:12:34.865059 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c308c-0f50-422c-abd0-4914ef32f4aa admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:12:34.925968 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c308c-0f50-422c-abd0-4914ef32f4aa admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:12:34.931795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53195062-3839-4f17-8f4e-f887af217c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:12:34.932660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53195062-3839-4f17-8f4e-f887af217c39 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:12:34.932963 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53195062-3839-4f17-8f4e-f887af217c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:12:34.933907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53195062-3839-4f17-8f4e-f887af217c39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:12:34.934764 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53195062-3839-4f17-8f4e-f887af217c39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:12:34.936812 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53195062-3839-4f17-8f4e-f887af217c39 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:12:34.966085 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1446/2855] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:12:34 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:12:34.967187 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9680d70-7f34-41a2-a01b-d4edd701b963 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:12:34.979156 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9680d70-7f34-41a2-a01b-d4edd701b963 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:12:35.034676 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03ca14-803e-44ba-bbff-f76f94efba9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:12:35.035292 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03ca14-803e-44ba-bbff-f76f94efba9c admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:12:35.035574 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03ca14-803e-44ba-bbff-f76f94efba9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:12:35.035851 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03ca14-803e-44ba-bbff-f76f94efba9c admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:12:35.036573 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd03ca14-803e-44ba-bbff-f76f94efba9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:12:35.038486 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd03ca14-803e-44ba-bbff-f76f94efba9c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:12:35.059714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1410/2856] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:12:34 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:12:35.063738 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53195062-3839-4f17-8f4e-f887af217c39 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:12:35.073881 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53195062-3839-4f17-8f4e-f887af217c39 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:12:35.078582 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf6261-03ca-4377-b876-1dfc67806c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:12:35.079179 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf6261-03ca-4377-b876-1dfc67806c73 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:12:35.079454 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf6261-03ca-4377-b876-1dfc67806c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:12:35.079725 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf6261-03ca-4377-b876-1dfc67806c73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:12:35.080451 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf6261-03ca-4377-b876-1dfc67806c73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:12:35.082017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf6261-03ca-4377-b876-1dfc67806c73 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:12:35.092276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1447/2857] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:12:35.096336 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03ca14-803e-44ba-bbff-f76f94efba9c admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:12:35.106835 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03ca14-803e-44ba-bbff-f76f94efba9c admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:12:35.111566 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0575ae7-40ef-45fe-bcb3-dfbb298c5143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:12:35.112153 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0575ae7-40ef-45fe-bcb3-dfbb298c5143 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:12:35.112475 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0575ae7-40ef-45fe-bcb3-dfbb298c5143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:12:35.112752 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0575ae7-40ef-45fe-bcb3-dfbb298c5143 admin admin] PATH_INFO: `/v3/projects/f8b2e01cfc1741f68c9f4969488a178a/users/b475f9b4f9c24cc5a7c74506011d8be9/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:12:35.127738 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0575ae7-40ef-45fe-bcb3-dfbb298c5143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8b2e01cfc1741f68c9f4969488a178a, user_id=b475f9b4f9c24cc5a7c74506011d8be9, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:12:35.130488 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0575ae7-40ef-45fe-bcb3-dfbb298c5143 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:12:35.143331 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1411/2858] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:12:35 2022] PUT /identity/v3/projects/f8b2e01cfc1741f68c9f4969488a178a/users/b475f9b4f9c24cc5a7c74506011d8be9/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:12:35.147286 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf6261-03ca-4377-b876-1dfc67806c73 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:12:35.157364 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf6261-03ca-4377-b876-1dfc67806c73 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:12:35.211334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb491fe0-d43c-43d7-b699-55e2a5efcc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:12:35.211950 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb491fe0-d43c-43d7-b699-55e2a5efcc43 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:12:35.212193 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb491fe0-d43c-43d7-b699-55e2a5efcc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:12:35.212495 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb491fe0-d43c-43d7-b699-55e2a5efcc43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:12:35.213171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb491fe0-d43c-43d7-b699-55e2a5efcc43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:12:35.215249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb491fe0-d43c-43d7-b699-55e2a5efcc43 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:12:35.225084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1448/2859] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:12:35.229243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0575ae7-40ef-45fe-bcb3-dfbb298c5143 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:12:35.240018 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0575ae7-40ef-45fe-bcb3-dfbb298c5143 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:12:35.244733 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f439dd1-18c9-45e6-b32e-c77fd6d9ded9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:12:35.245380 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f439dd1-18c9-45e6-b32e-c77fd6d9ded9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:12:35.245659 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f439dd1-18c9-45e6-b32e-c77fd6d9ded9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:12:35.245933 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f439dd1-18c9-45e6-b32e-c77fd6d9ded9 admin admin] PATH_INFO: `/v3/projects/f8b2e01cfc1741f68c9f4969488a178a/users/b475f9b4f9c24cc5a7c74506011d8be9/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:12:35.247857 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f439dd1-18c9-45e6-b32e-c77fd6d9ded9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8b2e01cfc1741f68c9f4969488a178a, user_id=b475f9b4f9c24cc5a7c74506011d8be9, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:12:35.250507 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f439dd1-18c9-45e6-b32e-c77fd6d9ded9 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:12:35.260393 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1412/2860] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:12:35 2022] PUT /identity/v3/projects/f8b2e01cfc1741f68c9f4969488a178a/users/b475f9b4f9c24cc5a7c74506011d8be9/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:12:35.270268 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b977b99-b4ee-4430-bdbc-c06b7f987293 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:12:35.270578 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b977b99-b4ee-4430-bdbc-c06b7f987293 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:12:35.270854 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b977b99-b4ee-4430-bdbc-c06b7f987293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:12:35.279505 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:12:35.301524 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5b977b99-b4ee-4430-bdbc-c06b7f987293 None None] MFA Rules not processed for user `b475f9b4f9c24cc5a7c74506011d8be9`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:12:35.302793 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5b977b99-b4ee-4430-bdbc-c06b7f987293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:12:35.373316 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1449/2861] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:12:35 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:12:35.384027 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f439dd1-18c9-45e6-b32e-c77fd6d9ded9 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:12:35.385708 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6f439dd1-18c9-45e6-b32e-c77fd6d9ded9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:12:35.399654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:12:35.445294 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f439dd1-18c9-45e6-b32e-c77fd6d9ded9 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:12:35.448971 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376859a5-bf35-4676-af89-8a7b6f28a354 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:12:35.449544 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376859a5-bf35-4676-af89-8a7b6f28a354 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:12:35.449906 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376859a5-bf35-4676-af89-8a7b6f28a354 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:12:35.450189 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376859a5-bf35-4676-af89-8a7b6f28a354 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:12:35.508136 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376859a5-bf35-4676-af89-8a7b6f28a354 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:12:35.509668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376859a5-bf35-4676-af89-8a7b6f28a354 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:12:35.518471 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1413/2862] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:12:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 06 13:12:42.196714 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b977b99-b4ee-4430-bdbc-c06b7f987293 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:12:42.206914 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b977b99-b4ee-4430-bdbc-c06b7f987293 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:12:42.210393 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9e483-5a13-4e6f-b88c-f7e54f261d21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:12:42.210988 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9e483-5a13-4e6f-b88c-f7e54f261d21 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:12:42.211269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9e483-5a13-4e6f-b88c-f7e54f261d21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:12:42.211535 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9e483-5a13-4e6f-b88c-f7e54f261d21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:12:42.220208 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9e483-5a13-4e6f-b88c-f7e54f261d21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:12:42.221964 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9e483-5a13-4e6f-b88c-f7e54f261d21 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:12:42.231207 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1450/2863] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:12:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:12:45.514432 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=43 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:12:49.432993 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376859a5-bf35-4676-af89-8a7b6f28a354 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:12:49.434885 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-376859a5-bf35-4676-af89-8a7b6f28a354 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:12:49.489626 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376859a5-bf35-4676-af89-8a7b6f28a354 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:12:49.493204 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14a4ac-a0f9-49d0-9e99-eaab6ec13535 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:12:49.493803 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14a4ac-a0f9-49d0-9e99-eaab6ec13535 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:12:49.494078 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14a4ac-a0f9-49d0-9e99-eaab6ec13535 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:12:49.494345 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14a4ac-a0f9-49d0-9e99-eaab6ec13535 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:12:49.496038 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-cc14a4ac-a0f9-49d0-9e99-eaab6ec13535 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:12:49.505152 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14a4ac-a0f9-49d0-9e99-eaab6ec13535 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:12:49.506561 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14a4ac-a0f9-49d0-9e99-eaab6ec13535 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:12:49.514259 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1414/2864] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:12:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:12:52.227146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=38 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:12:59.510788 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=26 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:13:13.350703 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9e483-5a13-4e6f-b88c-f7e54f261d21 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:13.412546 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9e483-5a13-4e6f-b88c-f7e54f261d21 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:13.417110 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d1c8a-bf6a-49fb-b55b-603fc5b9e34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:13.417733 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d1c8a-bf6a-49fb-b55b-603fc5b9e34c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:13.418008 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d1c8a-bf6a-49fb-b55b-603fc5b9e34c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:13.418301 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d1c8a-bf6a-49fb-b55b-603fc5b9e34c admin admin] PATH_INFO: `/v3/users/8b021907238445e1866c7dfa25894deb` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:13.419438 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d1c8a-bf6a-49fb-b55b-603fc5b9e34c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b021907238445e1866c7dfa25894deb)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:13.422034 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d1c8a-bf6a-49fb-b55b-603fc5b9e34c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:13.466085 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d95d1c8a-bf6a-49fb-b55b-603fc5b9e34c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b021907238445e1866c7dfa25894deb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:13.478472 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d95d1c8a-bf6a-49fb-b55b-603fc5b9e34c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b021907238445e1866c7dfa25894deb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:13.479126 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d95d1c8a-bf6a-49fb-b55b-603fc5b9e34c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b021907238445e1866c7dfa25894deb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:13.479728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d95d1c8a-bf6a-49fb-b55b-603fc5b9e34c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b021907238445e1866c7dfa25894deb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:13.487163 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d95d1c8a-bf6a-49fb-b55b-603fc5b9e34c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b021907238445e1866c7dfa25894deb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:13.499030 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1451/2865] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:13:13 2022] DELETE /identity/v3/users/8b021907238445e1866c7dfa25894deb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:13:13.503504 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36fe8f-095c-4388-a293-b9f9a6e2c5c2 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:13.504003 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36fe8f-095c-4388-a293-b9f9a6e2c5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:13.504105 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36fe8f-095c-4388-a293-b9f9a6e2c5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:13.522216 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-bc36fe8f-095c-4388-a293-b9f9a6e2c5c2 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:13:13.523388 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-bc36fe8f-095c-4388-a293-b9f9a6e2c5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:13.580160 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1415/2866] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:13:13.589741 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d1c8a-bf6a-49fb-b55b-603fc5b9e34c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:13.591146 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d95d1c8a-bf6a-49fb-b55b-603fc5b9e34c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:13.649268 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d1c8a-bf6a-49fb-b55b-603fc5b9e34c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:13.652865 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d801395-e8fb-4cd0-98e9-cd44e5cc25d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:13.653469 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d801395-e8fb-4cd0-98e9-cd44e5cc25d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:13.653745 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d801395-e8fb-4cd0-98e9-cd44e5cc25d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:13.654016 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d801395-e8fb-4cd0-98e9-cd44e5cc25d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:13.663601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d801395-e8fb-4cd0-98e9-cd44e5cc25d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:13.665078 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d801395-e8fb-4cd0-98e9-cd44e5cc25d8 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:13.674893 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1452/2867] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:13:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:13:13.936347 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36fe8f-095c-4388-a293-b9f9a6e2c5c2 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:13.938267 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-bc36fe8f-095c-4388-a293-b9f9a6e2c5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:13.950633 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36fe8f-095c-4388-a293-b9f9a6e2c5c2 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:13.956564 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffeb7d1-986e-4471-8698-c46f95118835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:13.957221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffeb7d1-986e-4471-8698-c46f95118835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:13.957503 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffeb7d1-986e-4471-8698-c46f95118835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:13.957782 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffeb7d1-986e-4471-8698-c46f95118835 admin admin] PATH_INFO: `/v3/projects/8eaf312d87f649d5a9d63909d248473d` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:13.959022 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffeb7d1-986e-4471-8698-c46f95118835 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eaf312d87f649d5a9d63909d248473d)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:13.960988 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffeb7d1-986e-4471-8698-c46f95118835 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:14.011026 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-5ffeb7d1-986e-4471-8698-c46f95118835 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eaf312d87f649d5a9d63909d248473d', 'request_id': 'req-5ffeb7d1-986e-4471-8698-c46f95118835'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:14.017058 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-5ffeb7d1-986e-4471-8698-c46f95118835 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eaf312d87f649d5a9d63909d248473d', 'request_id': 'req-5ffeb7d1-986e-4471-8698-c46f95118835'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:14.024243 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-5ffeb7d1-986e-4471-8698-c46f95118835 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eaf312d87f649d5a9d63909d248473d', 'request_id': 'req-5ffeb7d1-986e-4471-8698-c46f95118835'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:14.035613 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-5ffeb7d1-986e-4471-8698-c46f95118835 admin admin] The token cache is being invalidate because project 8eaf312d87f649d5a9d63909d248473d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:13:14.035898 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-5ffeb7d1-986e-4471-8698-c46f95118835 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:14.036542 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-5ffeb7d1-986e-4471-8698-c46f95118835 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:14.038033 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1416/2868] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:13:13 2022] DELETE /identity/v3/projects/8eaf312d87f649d5a9d63909d248473d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:13:14.042129 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d801395-e8fb-4cd0-98e9-cd44e5cc25d8 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:14.051851 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d801395-e8fb-4cd0-98e9-cd44e5cc25d8 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:14.070511 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:13:14.110510 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d152ee4-42b6-4a75-89dd-74eea2a267e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:14.111081 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d152ee4-42b6-4a75-89dd-74eea2a267e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:14.111359 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d152ee4-42b6-4a75-89dd-74eea2a267e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:14.111632 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d152ee4-42b6-4a75-89dd-74eea2a267e6 admin admin] PATH_INFO: `/v3/users/b475f9b4f9c24cc5a7c74506011d8be9` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:14.112802 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d152ee4-42b6-4a75-89dd-74eea2a267e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b475f9b4f9c24cc5a7c74506011d8be9)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:14.114708 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d152ee4-42b6-4a75-89dd-74eea2a267e6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:14.159375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1d152ee4-42b6-4a75-89dd-74eea2a267e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b475f9b4f9c24cc5a7c74506011d8be9'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:14.171512 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1d152ee4-42b6-4a75-89dd-74eea2a267e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b475f9b4f9c24cc5a7c74506011d8be9'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:14.172171 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1d152ee4-42b6-4a75-89dd-74eea2a267e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b475f9b4f9c24cc5a7c74506011d8be9'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:14.172809 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1d152ee4-42b6-4a75-89dd-74eea2a267e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b475f9b4f9c24cc5a7c74506011d8be9'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:14.180730 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1d152ee4-42b6-4a75-89dd-74eea2a267e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b475f9b4f9c24cc5a7c74506011d8be9'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:14.193236 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1453/2869] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:13:14 2022] DELETE /identity/v3/users/b475f9b4f9c24cc5a7c74506011d8be9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:13:14.455939 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffeb7d1-986e-4471-8698-c46f95118835 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:14.457433 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-5ffeb7d1-986e-4471-8698-c46f95118835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:14.472666 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffeb7d1-986e-4471-8698-c46f95118835 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:14.528934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340acdb1-61ec-446c-a09c-35f2c5622b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:14.529555 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340acdb1-61ec-446c-a09c-35f2c5622b59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:14.529828 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340acdb1-61ec-446c-a09c-35f2c5622b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:14.530100 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340acdb1-61ec-446c-a09c-35f2c5622b59 admin admin] PATH_INFO: `/v3/projects/f8b2e01cfc1741f68c9f4969488a178a` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:14.531275 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340acdb1-61ec-446c-a09c-35f2c5622b59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8b2e01cfc1741f68c9f4969488a178a)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:14.533214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340acdb1-61ec-446c-a09c-35f2c5622b59 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:14.563974 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:13:14.581552 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-340acdb1-61ec-446c-a09c-35f2c5622b59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8b2e01cfc1741f68c9f4969488a178a', 'request_id': 'req-340acdb1-61ec-446c-a09c-35f2c5622b59'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:14.587174 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-340acdb1-61ec-446c-a09c-35f2c5622b59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8b2e01cfc1741f68c9f4969488a178a', 'request_id': 'req-340acdb1-61ec-446c-a09c-35f2c5622b59'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:14.593982 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-340acdb1-61ec-446c-a09c-35f2c5622b59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8b2e01cfc1741f68c9f4969488a178a', 'request_id': 'req-340acdb1-61ec-446c-a09c-35f2c5622b59'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:14.604858 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-340acdb1-61ec-446c-a09c-35f2c5622b59 admin admin] The token cache is being invalidate because project f8b2e01cfc1741f68c9f4969488a178a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:13:14.605141 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-340acdb1-61ec-446c-a09c-35f2c5622b59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:14.605779 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-340acdb1-61ec-446c-a09c-35f2c5622b59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:14.607249 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1417/2870] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:13:14 2022] DELETE /identity/v3/projects/f8b2e01cfc1741f68c9f4969488a178a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:13:14.612706 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6fd4a-4f8e-4f15-b810-900ce1193ab4 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:14.613173 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6fd4a-4f8e-4f15-b810-900ce1193ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:14.613281 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6fd4a-4f8e-4f15-b810-900ce1193ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:14.632656 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a5c6fd4a-4f8e-4f15-b810-900ce1193ab4 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:13:14.633765 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a5c6fd4a-4f8e-4f15-b810-900ce1193ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:14.690331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1454/2871] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:13:14.699077 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e4920-6e0d-448d-8287-81d9f71ad36f None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:14.699387 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e4920-6e0d-448d-8287-81d9f71ad36f None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:14.699638 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e4920-6e0d-448d-8287-81d9f71ad36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:14.718698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-b69e4920-6e0d-448d-8287-81d9f71ad36f None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:13:14.719879 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-b69e4920-6e0d-448d-8287-81d9f71ad36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:14.728238 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1418/2872] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:13:14.732341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6fd4a-4f8e-4f15-b810-900ce1193ab4 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:14.742332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6fd4a-4f8e-4f15-b810-900ce1193ab4 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:14.746871 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff25242-5d32-40fc-832f-71adabf23035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:14.747466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff25242-5d32-40fc-832f-71adabf23035 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:14.747743 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff25242-5d32-40fc-832f-71adabf23035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:14.748013 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff25242-5d32-40fc-832f-71adabf23035 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:14.748728 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff25242-5d32-40fc-832f-71adabf23035 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:13:14.749044 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff25242-5d32-40fc-832f-71adabf23035 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:14.751246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff25242-5d32-40fc-832f-71adabf23035 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:14.762292 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1455/2873] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 13:13:14.766333 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e4920-6e0d-448d-8287-81d9f71ad36f None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:14.775374 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e4920-6e0d-448d-8287-81d9f71ad36f None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:14.780035 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34586d96-6f63-418c-80b8-53eb069a3fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:14.780671 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34586d96-6f63-418c-80b8-53eb069a3fca admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:14.780948 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34586d96-6f63-418c-80b8-53eb069a3fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:14.781220 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34586d96-6f63-418c-80b8-53eb069a3fca admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:14.781876 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34586d96-6f63-418c-80b8-53eb069a3fca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:14.783755 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34586d96-6f63-418c-80b8-53eb069a3fca admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:14.810225 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1419/2874] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:13:14 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:13:14.814713 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff25242-5d32-40fc-832f-71adabf23035 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:14.824489 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff25242-5d32-40fc-832f-71adabf23035 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:14.850594 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:13:14.883090 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75776831-0bea-48f7-b6f0-c180708fb5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:14.883629 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75776831-0bea-48f7-b6f0-c180708fb5b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:14.883911 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75776831-0bea-48f7-b6f0-c180708fb5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:14.884185 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75776831-0bea-48f7-b6f0-c180708fb5b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:14.884883 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75776831-0bea-48f7-b6f0-c180708fb5b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:14.886772 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75776831-0bea-48f7-b6f0-c180708fb5b4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:14.907492 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1456/2875] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:13:14 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:13:14.911601 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34586d96-6f63-418c-80b8-53eb069a3fca admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:14.922326 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34586d96-6f63-418c-80b8-53eb069a3fca admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:14.926880 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc38b47-8e28-4b4c-a598-19f22fe683bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:14.927483 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc38b47-8e28-4b4c-a598-19f22fe683bd admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:14.927762 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc38b47-8e28-4b4c-a598-19f22fe683bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:14.928032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc38b47-8e28-4b4c-a598-19f22fe683bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:14.928751 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc38b47-8e28-4b4c-a598-19f22fe683bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:14.930336 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc38b47-8e28-4b4c-a598-19f22fe683bd admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:14.940759 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1420/2876] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:13:14.944611 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75776831-0bea-48f7-b6f0-c180708fb5b4 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:14.954762 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75776831-0bea-48f7-b6f0-c180708fb5b4 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:14.959450 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925fcbb-a57a-40c7-94b6-3d5836fb650d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:14.960055 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925fcbb-a57a-40c7-94b6-3d5836fb650d admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:14.960374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925fcbb-a57a-40c7-94b6-3d5836fb650d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:14.960704 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925fcbb-a57a-40c7-94b6-3d5836fb650d admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:14.961465 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925fcbb-a57a-40c7-94b6-3d5836fb650d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:14.963096 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925fcbb-a57a-40c7-94b6-3d5836fb650d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:14.975249 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1457/2877] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 13:13:14.979648 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc38b47-8e28-4b4c-a598-19f22fe683bd admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:14.990368 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc38b47-8e28-4b4c-a598-19f22fe683bd admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:14.995021 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aac1a5-63a1-4980-9677-bf82918635ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:14.995678 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aac1a5-63a1-4980-9677-bf82918635ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:14.995956 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aac1a5-63a1-4980-9677-bf82918635ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:14.996223 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aac1a5-63a1-4980-9677-bf82918635ca admin admin] PATH_INFO: `/v3/projects/91544b1df0f84356833f82742a2b7e88/users/eb1fd591f4204172ad217b9d44e9e8f0/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:15.010790 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aac1a5-63a1-4980-9677-bf82918635ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=91544b1df0f84356833f82742a2b7e88, user_id=eb1fd591f4204172ad217b9d44e9e8f0, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:15.013491 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aac1a5-63a1-4980-9677-bf82918635ca admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:15.026138 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1421/2878] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:13:14 2022] PUT /identity/v3/projects/91544b1df0f84356833f82742a2b7e88/users/eb1fd591f4204172ad217b9d44e9e8f0/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:13:15.036448 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7facc4d-f6f2-4f2b-8d65-8436b34b93e4 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:15.036733 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7facc4d-f6f2-4f2b-8d65-8436b34b93e4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:15.037009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7facc4d-f6f2-4f2b-8d65-8436b34b93e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:15.064814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f7facc4d-f6f2-4f2b-8d65-8436b34b93e4 None None] MFA Rules not processed for user `eb1fd591f4204172ad217b9d44e9e8f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:13:15.066093 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f7facc4d-f6f2-4f2b-8d65-8436b34b93e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:15.129241 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1458/2879] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:13:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:13:15.133784 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aac1a5-63a1-4980-9677-bf82918635ca admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:15.193305 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aac1a5-63a1-4980-9677-bf82918635ca admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:15.197928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1fd57-399c-441f-a85b-f1a6d7516fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:15.198528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1fd57-399c-441f-a85b-f1a6d7516fb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:15.198807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1fd57-399c-441f-a85b-f1a6d7516fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:15.199084 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1fd57-399c-441f-a85b-f1a6d7516fb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:15.199745 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b1fd57-399c-441f-a85b-f1a6d7516fb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:15.201698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b1fd57-399c-441f-a85b-f1a6d7516fb0 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:15.228726 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1422/2880] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:13:15 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:13:15.232652 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7facc4d-f6f2-4f2b-8d65-8436b34b93e4 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:15.243246 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7facc4d-f6f2-4f2b-8d65-8436b34b93e4 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:15.298337 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a36b7-4745-42bd-8d65-d2e34a221d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:15.298945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a36b7-4745-42bd-8d65-d2e34a221d8a admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:15.299224 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a36b7-4745-42bd-8d65-d2e34a221d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:15.299497 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a36b7-4745-42bd-8d65-d2e34a221d8a admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:15.300158 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a36b7-4745-42bd-8d65-d2e34a221d8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:15.302080 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a36b7-4745-42bd-8d65-d2e34a221d8a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:15.328091 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1459/2881] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:13:15 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:13:15.332254 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1fd57-399c-441f-a85b-f1a6d7516fb0 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:15.342489 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1fd57-399c-441f-a85b-f1a6d7516fb0 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:15.347961 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd0336-b084-4033-af52-2a57e6d3eb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:15.348589 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd0336-b084-4033-af52-2a57e6d3eb9a admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:15.348867 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd0336-b084-4033-af52-2a57e6d3eb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:15.349140 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd0336-b084-4033-af52-2a57e6d3eb9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:15.349821 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd0336-b084-4033-af52-2a57e6d3eb9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:15.351403 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd0336-b084-4033-af52-2a57e6d3eb9a admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:15.361627 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1423/2882] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:13:15.365642 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a36b7-4745-42bd-8d65-d2e34a221d8a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:15.375542 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a36b7-4745-42bd-8d65-d2e34a221d8a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:15.380186 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78133f8a-3b82-4b8e-b68c-7d5631ab5eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:15.380814 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78133f8a-3b82-4b8e-b68c-7d5631ab5eab admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:15.381097 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78133f8a-3b82-4b8e-b68c-7d5631ab5eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:15.381369 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78133f8a-3b82-4b8e-b68c-7d5631ab5eab admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:15.382049 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78133f8a-3b82-4b8e-b68c-7d5631ab5eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:15.383621 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78133f8a-3b82-4b8e-b68c-7d5631ab5eab admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:15.393995 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1460/2883] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 13:13:15.397893 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd0336-b084-4033-af52-2a57e6d3eb9a admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:15.409370 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd0336-b084-4033-af52-2a57e6d3eb9a admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:15.414508 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993282e-dbb3-4ed6-8996-b2309fb5faef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:15.415145 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993282e-dbb3-4ed6-8996-b2309fb5faef admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:15.415421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993282e-dbb3-4ed6-8996-b2309fb5faef admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:15.415689 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993282e-dbb3-4ed6-8996-b2309fb5faef admin admin] PATH_INFO: `/v3/projects/8dd4597a19d445c78aebe4ae29ce12dd/users/38e20f12b4854f8a8afea9cf2c42fa4f/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:15.430264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993282e-dbb3-4ed6-8996-b2309fb5faef admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dd4597a19d445c78aebe4ae29ce12dd, user_id=38e20f12b4854f8a8afea9cf2c42fa4f, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:15.432972 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993282e-dbb3-4ed6-8996-b2309fb5faef admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:15.441476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:13:15.446440 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1424/2884] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:13:15 2022] PUT /identity/v3/projects/8dd4597a19d445c78aebe4ae29ce12dd/users/38e20f12b4854f8a8afea9cf2c42fa4f/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:13:15.457085 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cacfaf-7449-4e7f-b932-c5ee1d1deeca None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:15.457374 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cacfaf-7449-4e7f-b932-c5ee1d1deeca None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:15.457654 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cacfaf-7449-4e7f-b932-c5ee1d1deeca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:15.486664 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f8cacfaf-7449-4e7f-b932-c5ee1d1deeca None None] MFA Rules not processed for user `38e20f12b4854f8a8afea9cf2c42fa4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:13:15.487885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f8cacfaf-7449-4e7f-b932-c5ee1d1deeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:15.491768 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:13:15.550642 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1461/2885] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:13:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 06 13:13:15.555029 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993282e-dbb3-4ed6-8996-b2309fb5faef admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:15.614113 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993282e-dbb3-4ed6-8996-b2309fb5faef admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:15.618798 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3d78e-edc9-4e28-a5b6-19db65b2f7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:15.619392 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3d78e-edc9-4e28-a5b6-19db65b2f7ff admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:15.619668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3d78e-edc9-4e28-a5b6-19db65b2f7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:15.619937 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3d78e-edc9-4e28-a5b6-19db65b2f7ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:15.620615 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3d78e-edc9-4e28-a5b6-19db65b2f7ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:15.622500 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3d78e-edc9-4e28-a5b6-19db65b2f7ff admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:15.648354 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1425/2886] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:13:15 2022] POST /identity/v3/projects => generated 387 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:13:15.652322 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cacfaf-7449-4e7f-b932-c5ee1d1deeca None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:15.663427 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cacfaf-7449-4e7f-b932-c5ee1d1deeca None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:15.717994 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c547c8-5bd4-4cd1-b601-bd821d3bafd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:15.718601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c547c8-5bd4-4cd1-b601-bd821d3bafd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:15.718886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c547c8-5bd4-4cd1-b601-bd821d3bafd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:15.719161 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c547c8-5bd4-4cd1-b601-bd821d3bafd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:15.719819 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c547c8-5bd4-4cd1-b601-bd821d3bafd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:15.722038 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c547c8-5bd4-4cd1-b601-bd821d3bafd0 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:15.742477 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1462/2887] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:13:15 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:13:15.746395 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3d78e-edc9-4e28-a5b6-19db65b2f7ff admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:15.757010 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3d78e-edc9-4e28-a5b6-19db65b2f7ff admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:15.761535 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb0c63-6c6c-4bb1-83ff-eea7c29c0901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:15.762126 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb0c63-6c6c-4bb1-83ff-eea7c29c0901 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:15.762401 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb0c63-6c6c-4bb1-83ff-eea7c29c0901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:15.762668 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb0c63-6c6c-4bb1-83ff-eea7c29c0901 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:15.763378 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb0c63-6c6c-4bb1-83ff-eea7c29c0901 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:15.765725 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb0c63-6c6c-4bb1-83ff-eea7c29c0901 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:15.776086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1426/2888] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:13:15.780044 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c547c8-5bd4-4cd1-b601-bd821d3bafd0 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:15.790015 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c547c8-5bd4-4cd1-b601-bd821d3bafd0 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:15.794663 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e19642-6ad9-446c-84e2-79ce7a80a052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:15.795308 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e19642-6ad9-446c-84e2-79ce7a80a052 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:15.795592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e19642-6ad9-446c-84e2-79ce7a80a052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:15.795893 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e19642-6ad9-446c-84e2-79ce7a80a052 admin admin] PATH_INFO: `/v3/projects/33a743b5c41143aa8f58cad7ed36c035/users/886664dc519b4a459cd489b5ba6097b3/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:15.811425 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e19642-6ad9-446c-84e2-79ce7a80a052 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33a743b5c41143aa8f58cad7ed36c035, user_id=886664dc519b4a459cd489b5ba6097b3, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:15.814617 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e19642-6ad9-446c-84e2-79ce7a80a052 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:15.827872 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1463/2889] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:13:15 2022] PUT /identity/v3/projects/33a743b5c41143aa8f58cad7ed36c035/users/886664dc519b4a459cd489b5ba6097b3/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:13:15.831697 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb0c63-6c6c-4bb1-83ff-eea7c29c0901 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:15.841586 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb0c63-6c6c-4bb1-83ff-eea7c29c0901 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:15.897214 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbb39f-d89c-4aab-b0b9-05853ebf9720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:15.897818 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbb39f-d89c-4aab-b0b9-05853ebf9720 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:15.898099 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbb39f-d89c-4aab-b0b9-05853ebf9720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:15.898425 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbb39f-d89c-4aab-b0b9-05853ebf9720 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:15.899088 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbb39f-d89c-4aab-b0b9-05853ebf9720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:15.900740 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbb39f-d89c-4aab-b0b9-05853ebf9720 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:15.910260 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1427/2890] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:13:15.914287 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e19642-6ad9-446c-84e2-79ce7a80a052 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:15.924996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e19642-6ad9-446c-84e2-79ce7a80a052 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:15.929890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0579fd-4dbc-4446-8c66-8d881e7a4402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:15.930539 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0579fd-4dbc-4446-8c66-8d881e7a4402 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:15.930811 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0579fd-4dbc-4446-8c66-8d881e7a4402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:15.931088 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0579fd-4dbc-4446-8c66-8d881e7a4402 admin admin] PATH_INFO: `/v3/projects/33a743b5c41143aa8f58cad7ed36c035/users/886664dc519b4a459cd489b5ba6097b3/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:15.933202 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0579fd-4dbc-4446-8c66-8d881e7a4402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33a743b5c41143aa8f58cad7ed36c035, user_id=886664dc519b4a459cd489b5ba6097b3, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:15.935795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0579fd-4dbc-4446-8c66-8d881e7a4402 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:15.946628 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1464/2891] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:13:15 2022] PUT /identity/v3/projects/33a743b5c41143aa8f58cad7ed36c035/users/886664dc519b4a459cd489b5ba6097b3/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:13:15.957629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6184e75-b273-4296-b6dd-cf606d2f6e98 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:15.957929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6184e75-b273-4296-b6dd-cf606d2f6e98 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:15.958209 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6184e75-b273-4296-b6dd-cf606d2f6e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:15.986440 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-d6184e75-b273-4296-b6dd-cf606d2f6e98 None None] MFA Rules not processed for user `886664dc519b4a459cd489b5ba6097b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:13:15.987638 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d6184e75-b273-4296-b6dd-cf606d2f6e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:16.044699 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:13:16.054277 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1428/2892] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:13:15 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:13:16.065490 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0579fd-4dbc-4446-8c66-8d881e7a4402 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:16.067219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1c0579fd-4dbc-4446-8c66-8d881e7a4402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:16.130277 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:13:16.145333 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0579fd-4dbc-4446-8c66-8d881e7a4402 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:16.149006 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a73cc8e-e32f-45a9-9bb5-147f794d681e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:16.149546 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73cc8e-e32f-45a9-9bb5-147f794d681e service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:16.149820 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73cc8e-e32f-45a9-9bb5-147f794d681e service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:16.150092 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73cc8e-e32f-45a9-9bb5-147f794d681e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:16.158819 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a73cc8e-e32f-45a9-9bb5-147f794d681e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:16.160222 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a73cc8e-e32f-45a9-9bb5-147f794d681e service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:16.169510 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1465/2893] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:13:16 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:13:16.655938 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6184e75-b273-4296-b6dd-cf606d2f6e98 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:16.657349 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d6184e75-b273-4296-b6dd-cf606d2f6e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:16.669192 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6184e75-b273-4296-b6dd-cf606d2f6e98 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:16.673201 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5f925-92e8-4da0-ba6e-0e55cc9c89e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:16.673802 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5f925-92e8-4da0-ba6e-0e55cc9c89e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:16.674082 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5f925-92e8-4da0-ba6e-0e55cc9c89e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:16.674355 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5f925-92e8-4da0-ba6e-0e55cc9c89e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:16.729382 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5f925-92e8-4da0-ba6e-0e55cc9c89e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:16.730796 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5f925-92e8-4da0-ba6e-0e55cc9c89e7 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:16.739929 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1429/2894] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:13:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:13:20.724917 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a73cc8e-e32f-45a9-9bb5-147f794d681e service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:20.733387 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a73cc8e-e32f-45a9-9bb5-147f794d681e service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:20.737010 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9c6df-4987-45c6-b464-8e7b5d388296 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:20.737603 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9c6df-4987-45c6-b464-8e7b5d388296 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:20.737881 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9c6df-4987-45c6-b464-8e7b5d388296 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:20.738151 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9c6df-4987-45c6-b464-8e7b5d388296 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:20.794746 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb9c6df-4987-45c6-b464-8e7b5d388296 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:20.796167 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb9c6df-4987-45c6-b464-8e7b5d388296 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:20.804821 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1466/2895] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:13:20 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:13:22.844552 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5f925-92e8-4da0-ba6e-0e55cc9c89e7 service cinder] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:22.904762 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5f925-92e8-4da0-ba6e-0e55cc9c89e7 service cinder] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:22.909219 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a864e-2414-4537-ba33-1917fb23e19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:22.909840 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a864e-2414-4537-ba33-1917fb23e19f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:22.910118 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a864e-2414-4537-ba33-1917fb23e19f admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:22.910389 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a864e-2414-4537-ba33-1917fb23e19f admin admin] PATH_INFO: `/v3/users/eb1fd591f4204172ad217b9d44e9e8f0` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:22.911519 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a864e-2414-4537-ba33-1917fb23e19f admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb1fd591f4204172ad217b9d44e9e8f0)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:22.913457 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a864e-2414-4537-ba33-1917fb23e19f admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:22.958193 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-236a864e-2414-4537-ba33-1917fb23e19f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb1fd591f4204172ad217b9d44e9e8f0'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:22.965127 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-236a864e-2414-4537-ba33-1917fb23e19f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb1fd591f4204172ad217b9d44e9e8f0'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:22.977220 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-236a864e-2414-4537-ba33-1917fb23e19f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb1fd591f4204172ad217b9d44e9e8f0'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:22.977913 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-236a864e-2414-4537-ba33-1917fb23e19f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb1fd591f4204172ad217b9d44e9e8f0'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:22.978509 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-236a864e-2414-4537-ba33-1917fb23e19f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb1fd591f4204172ad217b9d44e9e8f0'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:22.991461 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1430/2896] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:13:22 2022] DELETE /identity/v3/users/eb1fd591f4204172ad217b9d44e9e8f0 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:13:22.996063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b41c04-7d3c-4a9a-ac90-9a02403aa573 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:22.996584 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b41c04-7d3c-4a9a-ac90-9a02403aa573 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:22.996709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b41c04-7d3c-4a9a-ac90-9a02403aa573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:23.016214 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c4b41c04-7d3c-4a9a-ac90-9a02403aa573 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:13:23.017365 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c4b41c04-7d3c-4a9a-ac90-9a02403aa573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:23.074207 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1467/2897] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:13:23.084015 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a864e-2414-4537-ba33-1917fb23e19f admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:23.085480 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-236a864e-2414-4537-ba33-1917fb23e19f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:23.100155 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:13:23.144024 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a864e-2414-4537-ba33-1917fb23e19f admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:23.147734 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d5fe2-4ded-4df1-bff0-629d8551cd7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:23.148372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d5fe2-4ded-4df1-bff0-629d8551cd7a service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:23.148649 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d5fe2-4ded-4df1-bff0-629d8551cd7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:23.148916 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d5fe2-4ded-4df1-bff0-629d8551cd7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:23.158392 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d5fe2-4ded-4df1-bff0-629d8551cd7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:23.159804 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d5fe2-4ded-4df1-bff0-629d8551cd7a service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:23.169090 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1431/2898] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:13:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:13:23.432069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b41c04-7d3c-4a9a-ac90-9a02403aa573 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:23.433532 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c4b41c04-7d3c-4a9a-ac90-9a02403aa573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:23.446670 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b41c04-7d3c-4a9a-ac90-9a02403aa573 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:23.451784 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306390c-7749-4c79-926a-81799541254c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:23.452464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306390c-7749-4c79-926a-81799541254c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:23.452771 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306390c-7749-4c79-926a-81799541254c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:23.453523 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306390c-7749-4c79-926a-81799541254c admin admin] PATH_INFO: `/v3/projects/91544b1df0f84356833f82742a2b7e88` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:23.454693 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f306390c-7749-4c79-926a-81799541254c admin admin] RBAC: Authorizing `identity:delete_project(project_id=91544b1df0f84356833f82742a2b7e88)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:23.456649 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f306390c-7749-4c79-926a-81799541254c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:23.503157 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f306390c-7749-4c79-926a-81799541254c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91544b1df0f84356833f82742a2b7e88', 'request_id': 'req-f306390c-7749-4c79-926a-81799541254c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:23.508530 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f306390c-7749-4c79-926a-81799541254c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91544b1df0f84356833f82742a2b7e88', 'request_id': 'req-f306390c-7749-4c79-926a-81799541254c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:23.516266 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f306390c-7749-4c79-926a-81799541254c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91544b1df0f84356833f82742a2b7e88', 'request_id': 'req-f306390c-7749-4c79-926a-81799541254c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:23.527115 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f306390c-7749-4c79-926a-81799541254c admin admin] The token cache is being invalidate because project 91544b1df0f84356833f82742a2b7e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:13:23.527399 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f306390c-7749-4c79-926a-81799541254c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:23.528025 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-f306390c-7749-4c79-926a-81799541254c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:23.529548 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1468/2899] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:13:23 2022] DELETE /identity/v3/projects/91544b1df0f84356833f82742a2b7e88 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:13:23.534152 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d5fe2-4ded-4df1-bff0-629d8551cd7a service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:23.543614 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d5fe2-4ded-4df1-bff0-629d8551cd7a service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:23.606743 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b73972-9b9a-4623-8644-36e3eeba6af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:23.607297 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b73972-9b9a-4623-8644-36e3eeba6af2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:23.607568 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b73972-9b9a-4623-8644-36e3eeba6af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:23.607844 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b73972-9b9a-4623-8644-36e3eeba6af2 admin admin] PATH_INFO: `/v3/users/38e20f12b4854f8a8afea9cf2c42fa4f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:23.609002 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b73972-9b9a-4623-8644-36e3eeba6af2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38e20f12b4854f8a8afea9cf2c42fa4f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:23.611581 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b73972-9b9a-4623-8644-36e3eeba6af2 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:23.655533 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f1b73972-9b9a-4623-8644-36e3eeba6af2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38e20f12b4854f8a8afea9cf2c42fa4f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:23.662319 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f1b73972-9b9a-4623-8644-36e3eeba6af2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38e20f12b4854f8a8afea9cf2c42fa4f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:23.674832 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f1b73972-9b9a-4623-8644-36e3eeba6af2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38e20f12b4854f8a8afea9cf2c42fa4f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:23.675501 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f1b73972-9b9a-4623-8644-36e3eeba6af2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38e20f12b4854f8a8afea9cf2c42fa4f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:23.676124 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f1b73972-9b9a-4623-8644-36e3eeba6af2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38e20f12b4854f8a8afea9cf2c42fa4f'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:23.688898 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1432/2900] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:13:23 2022] DELETE /identity/v3/users/38e20f12b4854f8a8afea9cf2c42fa4f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:13:23.954107 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306390c-7749-4c79-926a-81799541254c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:23.955527 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f306390c-7749-4c79-926a-81799541254c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:23.963467 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:13:23.971485 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306390c-7749-4c79-926a-81799541254c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:24.037686 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91b065-aafb-433c-afd3-4392202e4fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:24.038351 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91b065-aafb-433c-afd3-4392202e4fea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:24.038628 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91b065-aafb-433c-afd3-4392202e4fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:24.038899 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91b065-aafb-433c-afd3-4392202e4fea admin admin] PATH_INFO: `/v3/projects/8dd4597a19d445c78aebe4ae29ce12dd` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:24.040042 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d91b065-aafb-433c-afd3-4392202e4fea admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dd4597a19d445c78aebe4ae29ce12dd)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:24.041956 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d91b065-aafb-433c-afd3-4392202e4fea admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:24.087650 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7d91b065-aafb-433c-afd3-4392202e4fea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dd4597a19d445c78aebe4ae29ce12dd', 'request_id': 'req-7d91b065-aafb-433c-afd3-4392202e4fea'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:24.093006 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7d91b065-aafb-433c-afd3-4392202e4fea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dd4597a19d445c78aebe4ae29ce12dd', 'request_id': 'req-7d91b065-aafb-433c-afd3-4392202e4fea'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:24.100489 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7d91b065-aafb-433c-afd3-4392202e4fea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dd4597a19d445c78aebe4ae29ce12dd', 'request_id': 'req-7d91b065-aafb-433c-afd3-4392202e4fea'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:24.111595 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7d91b065-aafb-433c-afd3-4392202e4fea admin admin] The token cache is being invalidate because project 8dd4597a19d445c78aebe4ae29ce12dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:13:24.111886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7d91b065-aafb-433c-afd3-4392202e4fea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:24.112570 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7d91b065-aafb-433c-afd3-4392202e4fea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:24.114084 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1469/2901] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:13:23 2022] DELETE /identity/v3/projects/8dd4597a19d445c78aebe4ae29ce12dd => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:13:24.118108 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b73972-9b9a-4623-8644-36e3eeba6af2 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:24.119514 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f1b73972-9b9a-4623-8644-36e3eeba6af2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:24.132621 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b73972-9b9a-4623-8644-36e3eeba6af2 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:24.164107 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:13:24.192737 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deac176-8fbe-4a15-b4d9-dc303bfbeb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:24.193369 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deac176-8fbe-4a15-b4d9-dc303bfbeb67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:24.193654 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deac176-8fbe-4a15-b4d9-dc303bfbeb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:24.193930 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deac176-8fbe-4a15-b4d9-dc303bfbeb67 admin admin] PATH_INFO: `/v3/users/886664dc519b4a459cd489b5ba6097b3` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:24.195108 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deac176-8fbe-4a15-b4d9-dc303bfbeb67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=886664dc519b4a459cd489b5ba6097b3)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:24.197063 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deac176-8fbe-4a15-b4d9-dc303bfbeb67 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:24.241547 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8deac176-8fbe-4a15-b4d9-dc303bfbeb67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '886664dc519b4a459cd489b5ba6097b3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:24.250093 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8deac176-8fbe-4a15-b4d9-dc303bfbeb67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '886664dc519b4a459cd489b5ba6097b3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:24.272758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8deac176-8fbe-4a15-b4d9-dc303bfbeb67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '886664dc519b4a459cd489b5ba6097b3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:24.273601 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8deac176-8fbe-4a15-b4d9-dc303bfbeb67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '886664dc519b4a459cd489b5ba6097b3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:24.274215 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-8deac176-8fbe-4a15-b4d9-dc303bfbeb67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '886664dc519b4a459cd489b5ba6097b3'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:24.286281 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1433/2902] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:13:24 2022] DELETE /identity/v3/users/886664dc519b4a459cd489b5ba6097b3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:13:24.560499 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91b065-aafb-433c-afd3-4392202e4fea admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:24.562017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7d91b065-aafb-433c-afd3-4392202e4fea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:24.575514 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91b065-aafb-433c-afd3-4392202e4fea admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:24.635293 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f325e0-9c9f-4b04-9c8f-b66978bebd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:24.635843 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f325e0-9c9f-4b04-9c8f-b66978bebd5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:24.636119 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f325e0-9c9f-4b04-9c8f-b66978bebd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:24.636426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f325e0-9c9f-4b04-9c8f-b66978bebd5d admin admin] PATH_INFO: `/v3/projects/33a743b5c41143aa8f58cad7ed36c035` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:24.637578 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f325e0-9c9f-4b04-9c8f-b66978bebd5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=33a743b5c41143aa8f58cad7ed36c035)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:24.639468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f325e0-9c9f-4b04-9c8f-b66978bebd5d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:24.676592 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:13:24.688387 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d4f325e0-9c9f-4b04-9c8f-b66978bebd5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33a743b5c41143aa8f58cad7ed36c035', 'request_id': 'req-d4f325e0-9c9f-4b04-9c8f-b66978bebd5d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:24.693798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d4f325e0-9c9f-4b04-9c8f-b66978bebd5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33a743b5c41143aa8f58cad7ed36c035', 'request_id': 'req-d4f325e0-9c9f-4b04-9c8f-b66978bebd5d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:24.701073 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d4f325e0-9c9f-4b04-9c8f-b66978bebd5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33a743b5c41143aa8f58cad7ed36c035', 'request_id': 'req-d4f325e0-9c9f-4b04-9c8f-b66978bebd5d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:24.712173 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d4f325e0-9c9f-4b04-9c8f-b66978bebd5d admin admin] The token cache is being invalidate because project 33a743b5c41143aa8f58cad7ed36c035 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:13:24.712495 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d4f325e0-9c9f-4b04-9c8f-b66978bebd5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:24.713125 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d4f325e0-9c9f-4b04-9c8f-b66978bebd5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:24.714634 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1470/2903] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:13:24 2022] DELETE /identity/v3/projects/33a743b5c41143aa8f58cad7ed36c035 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:13:24.720527 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200562b-1973-4f84-9673-d03730929ebc None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:24.721069 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200562b-1973-4f84-9673-d03730929ebc None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:24.722165 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200562b-1973-4f84-9673-d03730929ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:24.741625 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-2200562b-1973-4f84-9673-d03730929ebc None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:13:24.742807 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2200562b-1973-4f84-9673-d03730929ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:24.800594 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1434/2904] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:13:24.808889 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e329b77-0355-4265-9ad0-7d27c588d2fe None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:24.809199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e329b77-0355-4265-9ad0-7d27c588d2fe None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:24.809480 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e329b77-0355-4265-9ad0-7d27c588d2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:24.828828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4e329b77-0355-4265-9ad0-7d27c588d2fe None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:13:24.830017 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4e329b77-0355-4265-9ad0-7d27c588d2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:24.838343 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1471/2905] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:13:24.842373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2200562b-1973-4f84-9673-d03730929ebc None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:24.852861 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2200562b-1973-4f84-9673-d03730929ebc None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:24.857461 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5363f1-198b-4f2b-b47d-d8141b6a6c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:24.858055 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5363f1-198b-4f2b-b47d-d8141b6a6c73 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:24.858330 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5363f1-198b-4f2b-b47d-d8141b6a6c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:24.858598 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5363f1-198b-4f2b-b47d-d8141b6a6c73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:24.859277 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5363f1-198b-4f2b-b47d-d8141b6a6c73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:13:24.859577 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5363f1-198b-4f2b-b47d-d8141b6a6c73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:24.861230 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5363f1-198b-4f2b-b47d-d8141b6a6c73 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:24.872652 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1435/2906] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 13:13:24.876714 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e329b77-0355-4265-9ad0-7d27c588d2fe None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:24.886213 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e329b77-0355-4265-9ad0-7d27c588d2fe None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:24.890771 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2e3db-abcd-433d-823f-752edfbfd25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:24.891370 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2e3db-abcd-433d-823f-752edfbfd25c admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:24.891656 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2e3db-abcd-433d-823f-752edfbfd25c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:24.891931 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2e3db-abcd-433d-823f-752edfbfd25c admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:24.892635 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c2e3db-abcd-433d-823f-752edfbfd25c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:24.895009 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c2e3db-abcd-433d-823f-752edfbfd25c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:24.929036 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1472/2907] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:13:24 2022] POST /identity/v3/projects => generated 389 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:13:24.933617 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5363f1-198b-4f2b-b47d-d8141b6a6c73 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:24.944610 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5363f1-198b-4f2b-b47d-d8141b6a6c73 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:24.986662 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:13:25.009239 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16e41d-4b1f-4c9f-9775-fca993a47d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:25.009859 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16e41d-4b1f-4c9f-9775-fca993a47d2c admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:25.010169 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16e41d-4b1f-4c9f-9775-fca993a47d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:25.010445 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16e41d-4b1f-4c9f-9775-fca993a47d2c admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:25.011109 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16e41d-4b1f-4c9f-9775-fca993a47d2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:25.013075 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16e41d-4b1f-4c9f-9775-fca993a47d2c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:25.034669 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1436/2908] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:13:24 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:13:25.038817 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2e3db-abcd-433d-823f-752edfbfd25c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:25.049035 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2e3db-abcd-433d-823f-752edfbfd25c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:25.053737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be7490-97d2-43e0-a5bd-e2f36ebdf87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:25.054335 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be7490-97d2-43e0-a5bd-e2f36ebdf87c admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:25.054617 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be7490-97d2-43e0-a5bd-e2f36ebdf87c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:25.054892 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be7490-97d2-43e0-a5bd-e2f36ebdf87c admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:25.055582 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be7490-97d2-43e0-a5bd-e2f36ebdf87c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:25.057204 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be7490-97d2-43e0-a5bd-e2f36ebdf87c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:25.066928 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1473/2909] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:13:25.070683 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16e41d-4b1f-4c9f-9775-fca993a47d2c admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:25.081370 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16e41d-4b1f-4c9f-9775-fca993a47d2c admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:25.085984 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a862b50-9721-41f6-8a2d-b4e45863251e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:25.086583 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a862b50-9721-41f6-8a2d-b4e45863251e admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:25.086863 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a862b50-9721-41f6-8a2d-b4e45863251e admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:25.087135 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a862b50-9721-41f6-8a2d-b4e45863251e admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:25.087817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a862b50-9721-41f6-8a2d-b4e45863251e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:25.089436 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a862b50-9721-41f6-8a2d-b4e45863251e admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:25.100580 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1437/2910] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:13:25.104736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be7490-97d2-43e0-a5bd-e2f36ebdf87c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:25.114791 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be7490-97d2-43e0-a5bd-e2f36ebdf87c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:25.119331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b044d9-abda-4892-9719-0bc352c24f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:25.119967 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b044d9-abda-4892-9719-0bc352c24f13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:25.120241 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b044d9-abda-4892-9719-0bc352c24f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:25.120542 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b044d9-abda-4892-9719-0bc352c24f13 admin admin] PATH_INFO: `/v3/projects/f32734bfd7b94cc8ae00a69f9eeaf0ec/users/b656e1ecde67488e91124567482a1f10/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:25.135199 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b044d9-abda-4892-9719-0bc352c24f13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f32734bfd7b94cc8ae00a69f9eeaf0ec, user_id=b656e1ecde67488e91124567482a1f10, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:25.137941 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b044d9-abda-4892-9719-0bc352c24f13 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:25.151057 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1474/2911] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:13:25 2022] PUT /identity/v3/projects/f32734bfd7b94cc8ae00a69f9eeaf0ec/users/b656e1ecde67488e91124567482a1f10/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:13:25.163918 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4b02e-7128-4cda-85e7-39069565a69a None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:25.164450 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4b02e-7128-4cda-85e7-39069565a69a None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:25.164566 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4b02e-7128-4cda-85e7-39069565a69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:25.193080 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-24c4b02e-7128-4cda-85e7-39069565a69a None None] MFA Rules not processed for user `b656e1ecde67488e91124567482a1f10`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:13:25.194284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-24c4b02e-7128-4cda-85e7-39069565a69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:25.265074 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1438/2912] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:13:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:13:25.270470 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b044d9-abda-4892-9719-0bc352c24f13 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:25.329913 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b044d9-abda-4892-9719-0bc352c24f13 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:25.335077 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0728dd-33c2-4923-8c17-a6dd3d41bf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:25.335680 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0728dd-33c2-4923-8c17-a6dd3d41bf59 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:25.335961 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0728dd-33c2-4923-8c17-a6dd3d41bf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:25.336238 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0728dd-33c2-4923-8c17-a6dd3d41bf59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:25.336919 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0728dd-33c2-4923-8c17-a6dd3d41bf59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:25.338806 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0728dd-33c2-4923-8c17-a6dd3d41bf59 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:25.364610 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1475/2913] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:13:25 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:13:25.368672 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4b02e-7128-4cda-85e7-39069565a69a None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:25.380134 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4b02e-7128-4cda-85e7-39069565a69a None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:25.434364 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60923239-dc43-4694-a1b9-1cdac1cb56a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:25.434938 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60923239-dc43-4694-a1b9-1cdac1cb56a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:25.435221 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60923239-dc43-4694-a1b9-1cdac1cb56a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:25.435496 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60923239-dc43-4694-a1b9-1cdac1cb56a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:25.436166 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60923239-dc43-4694-a1b9-1cdac1cb56a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:25.438123 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60923239-dc43-4694-a1b9-1cdac1cb56a7 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:25.459803 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1439/2914] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:13:25 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:13:25.463831 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0728dd-33c2-4923-8c17-a6dd3d41bf59 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:25.474537 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0728dd-33c2-4923-8c17-a6dd3d41bf59 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:25.479260 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829469c-a67d-4bdf-a6b0-c6943e33baf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:25.479867 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829469c-a67d-4bdf-a6b0-c6943e33baf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:25.480150 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829469c-a67d-4bdf-a6b0-c6943e33baf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:25.480466 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829469c-a67d-4bdf-a6b0-c6943e33baf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:25.481159 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829469c-a67d-4bdf-a6b0-c6943e33baf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:25.482929 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829469c-a67d-4bdf-a6b0-c6943e33baf7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:25.493631 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1476/2915] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:13:25.497696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60923239-dc43-4694-a1b9-1cdac1cb56a7 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:25.507998 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60923239-dc43-4694-a1b9-1cdac1cb56a7 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:25.512635 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85297e-0f78-4fde-99b0-f47bcf67796c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:25.513280 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85297e-0f78-4fde-99b0-f47bcf67796c admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:25.513563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85297e-0f78-4fde-99b0-f47bcf67796c admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:25.513866 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85297e-0f78-4fde-99b0-f47bcf67796c admin admin] PATH_INFO: `/v3/projects/8f7a705a24cd41cf9fb1880c50c746b1/users/f237626a5d214b11807c0b0fd04173a1/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:25.529072 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85297e-0f78-4fde-99b0-f47bcf67796c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f7a705a24cd41cf9fb1880c50c746b1, user_id=f237626a5d214b11807c0b0fd04173a1, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:25.531714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85297e-0f78-4fde-99b0-f47bcf67796c admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:25.547284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1440/2916] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:13:25 2022] PUT /identity/v3/projects/8f7a705a24cd41cf9fb1880c50c746b1/users/f237626a5d214b11807c0b0fd04173a1/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:13:25.551445 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829469c-a67d-4bdf-a6b0-c6943e33baf7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:25.564275 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829469c-a67d-4bdf-a6b0-c6943e33baf7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:25.595597 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:13:25.623733 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf160828-4a6c-4590-98ba-31cfe3be7756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:25.624381 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf160828-4a6c-4590-98ba-31cfe3be7756 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:25.624669 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf160828-4a6c-4590-98ba-31cfe3be7756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:25.624945 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf160828-4a6c-4590-98ba-31cfe3be7756 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:25.625636 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf160828-4a6c-4590-98ba-31cfe3be7756 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:25.627225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf160828-4a6c-4590-98ba-31cfe3be7756 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:25.637453 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1477/2917] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:13:25.641615 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85297e-0f78-4fde-99b0-f47bcf67796c admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:25.651934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85297e-0f78-4fde-99b0-f47bcf67796c admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:25.656548 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263fa8d-412c-45f1-80d1-87e2f8b9b4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:25.657224 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263fa8d-412c-45f1-80d1-87e2f8b9b4fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:25.657501 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263fa8d-412c-45f1-80d1-87e2f8b9b4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:25.657779 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263fa8d-412c-45f1-80d1-87e2f8b9b4fe admin admin] PATH_INFO: `/v3/projects/8f7a705a24cd41cf9fb1880c50c746b1/users/f237626a5d214b11807c0b0fd04173a1/roles/144c8425688b409384e1e8ed8279d88e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:25.659716 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263fa8d-412c-45f1-80d1-87e2f8b9b4fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f7a705a24cd41cf9fb1880c50c746b1, user_id=f237626a5d214b11807c0b0fd04173a1, role_id=144c8425688b409384e1e8ed8279d88e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:25.662363 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263fa8d-412c-45f1-80d1-87e2f8b9b4fe admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:25.672272 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1441/2918] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:13:25 2022] PUT /identity/v3/projects/8f7a705a24cd41cf9fb1880c50c746b1/users/f237626a5d214b11807c0b0fd04173a1/roles/144c8425688b409384e1e8ed8279d88e => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:13:25.682874 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee598c-b50f-4bce-8ad5-d7a414096b12 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:25.683305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee598c-b50f-4bce-8ad5-d7a414096b12 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:25.683576 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee598c-b50f-4bce-8ad5-d7a414096b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:25.711531 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-eeee598c-b50f-4bce-8ad5-d7a414096b12 None None] MFA Rules not processed for user `f237626a5d214b11807c0b0fd04173a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:13:25.712736 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-eeee598c-b50f-4bce-8ad5-d7a414096b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:25.778426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1478/2919] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:13:25 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:13:25.789053 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263fa8d-412c-45f1-80d1-87e2f8b9b4fe admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:25.790750 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f263fa8d-412c-45f1-80d1-87e2f8b9b4fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:25.796268 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:13:25.850264 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263fa8d-412c-45f1-80d1-87e2f8b9b4fe admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:25.855384 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88ab6d-5062-4c61-90e5-dafce7aff677 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:25.856362 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88ab6d-5062-4c61-90e5-dafce7aff677 service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:25.858106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88ab6d-5062-4c61-90e5-dafce7aff677 service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:25.858106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88ab6d-5062-4c61-90e5-dafce7aff677 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:25.866948 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88ab6d-5062-4c61-90e5-dafce7aff677 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:25.868404 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88ab6d-5062-4c61-90e5-dafce7aff677 service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:25.878864 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1442/2920] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:13:25 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:13:26.012520 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee598c-b50f-4bce-8ad5-d7a414096b12 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:26.072178 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee598c-b50f-4bce-8ad5-d7a414096b12 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:26.076766 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54dc13-af0a-481c-8f9c-a4bff2adbb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:26.077418 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54dc13-af0a-481c-8f9c-a4bff2adbb22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:26.077692 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54dc13-af0a-481c-8f9c-a4bff2adbb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:26.077957 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54dc13-af0a-481c-8f9c-a4bff2adbb22 admin admin] PATH_INFO: `/v3/users/b656e1ecde67488e91124567482a1f10` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:26.079101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54dc13-af0a-481c-8f9c-a4bff2adbb22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b656e1ecde67488e91124567482a1f10)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:26.081053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54dc13-af0a-481c-8f9c-a4bff2adbb22 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:26.128259 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-dd54dc13-af0a-481c-8f9c-a4bff2adbb22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b656e1ecde67488e91124567482a1f10'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:26.135185 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:13:26.141277 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-dd54dc13-af0a-481c-8f9c-a4bff2adbb22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b656e1ecde67488e91124567482a1f10'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:26.142031 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-dd54dc13-af0a-481c-8f9c-a4bff2adbb22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b656e1ecde67488e91124567482a1f10'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:26.142638 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-dd54dc13-af0a-481c-8f9c-a4bff2adbb22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b656e1ecde67488e91124567482a1f10'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:26.150573 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-dd54dc13-af0a-481c-8f9c-a4bff2adbb22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b656e1ecde67488e91124567482a1f10'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:26.163247 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1479/2921] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:13:26 2022] DELETE /identity/v3/users/b656e1ecde67488e91124567482a1f10 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:13:26.167840 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd98cd-2bb2-4fe5-9e58-747143131511 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:26.168334 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd98cd-2bb2-4fe5-9e58-747143131511 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:26.168442 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd98cd-2bb2-4fe5-9e58-747143131511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:26.187397 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-aecd98cd-2bb2-4fe5-9e58-747143131511 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:13:26.188617 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-aecd98cd-2bb2-4fe5-9e58-747143131511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:26.245415 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1443/2922] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:13:26.255388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54dc13-af0a-481c-8f9c-a4bff2adbb22 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:26.256797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-dd54dc13-af0a-481c-8f9c-a4bff2adbb22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:26.312540 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54dc13-af0a-481c-8f9c-a4bff2adbb22 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:26.316885 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cecd6de-4762-484b-a14f-9b31d1bdde45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:26.317492 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecd6de-4762-484b-a14f-9b31d1bdde45 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:26.317770 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecd6de-4762-484b-a14f-9b31d1bdde45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:26.318043 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecd6de-4762-484b-a14f-9b31d1bdde45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:26.326915 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cecd6de-4762-484b-a14f-9b31d1bdde45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:26.328359 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cecd6de-4762-484b-a14f-9b31d1bdde45 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:26.338457 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1480/2923] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:13:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:13:26.598128 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd98cd-2bb2-4fe5-9e58-747143131511 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:26.599559 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-aecd98cd-2bb2-4fe5-9e58-747143131511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:26.612629 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd98cd-2bb2-4fe5-9e58-747143131511 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:26.617268 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690b8a3-26aa-458f-824e-51addd4c38d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:26.617906 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690b8a3-26aa-458f-824e-51addd4c38d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:26.618183 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690b8a3-26aa-458f-824e-51addd4c38d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:26.618453 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690b8a3-26aa-458f-824e-51addd4c38d2 admin admin] PATH_INFO: `/v3/projects/f32734bfd7b94cc8ae00a69f9eeaf0ec` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:26.619605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690b8a3-26aa-458f-824e-51addd4c38d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f32734bfd7b94cc8ae00a69f9eeaf0ec)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:26.621537 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4690b8a3-26aa-458f-824e-51addd4c38d2 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:26.667444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4690b8a3-26aa-458f-824e-51addd4c38d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f32734bfd7b94cc8ae00a69f9eeaf0ec', 'request_id': 'req-4690b8a3-26aa-458f-824e-51addd4c38d2'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:26.672869 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4690b8a3-26aa-458f-824e-51addd4c38d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f32734bfd7b94cc8ae00a69f9eeaf0ec', 'request_id': 'req-4690b8a3-26aa-458f-824e-51addd4c38d2'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:26.681396 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4690b8a3-26aa-458f-824e-51addd4c38d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f32734bfd7b94cc8ae00a69f9eeaf0ec', 'request_id': 'req-4690b8a3-26aa-458f-824e-51addd4c38d2'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:26.692981 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4690b8a3-26aa-458f-824e-51addd4c38d2 admin admin] The token cache is being invalidate because project f32734bfd7b94cc8ae00a69f9eeaf0ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:13:26.693262 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4690b8a3-26aa-458f-824e-51addd4c38d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:26.693881 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-4690b8a3-26aa-458f-824e-51addd4c38d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:26.695376 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1444/2924] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:13:26 2022] DELETE /identity/v3/projects/f32734bfd7b94cc8ae00a69f9eeaf0ec => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:13:26.699271 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cecd6de-4762-484b-a14f-9b31d1bdde45 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:26.709949 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cecd6de-4762-484b-a14f-9b31d1bdde45 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:26.765203 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d758151-c4e7-4cab-abdd-be213a18879a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:26.765828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d758151-c4e7-4cab-abdd-be213a18879a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:26.766105 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d758151-c4e7-4cab-abdd-be213a18879a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:26.766375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d758151-c4e7-4cab-abdd-be213a18879a admin admin] PATH_INFO: `/v3/users/f237626a5d214b11807c0b0fd04173a1` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:26.767513 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d758151-c4e7-4cab-abdd-be213a18879a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f237626a5d214b11807c0b0fd04173a1)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:26.769474 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d758151-c4e7-4cab-abdd-be213a18879a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:26.814464 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5d758151-c4e7-4cab-abdd-be213a18879a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f237626a5d214b11807c0b0fd04173a1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:26.826062 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5d758151-c4e7-4cab-abdd-be213a18879a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f237626a5d214b11807c0b0fd04173a1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:26.826706 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5d758151-c4e7-4cab-abdd-be213a18879a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f237626a5d214b11807c0b0fd04173a1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:26.827305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5d758151-c4e7-4cab-abdd-be213a18879a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f237626a5d214b11807c0b0fd04173a1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:26.834235 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5d758151-c4e7-4cab-abdd-be213a18879a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f237626a5d214b11807c0b0fd04173a1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:26.839435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:13:26.848365 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1481/2925] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:13:26 2022] DELETE /identity/v3/users/f237626a5d214b11807c0b0fd04173a1 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:13:27.123598 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690b8a3-26aa-458f-824e-51addd4c38d2 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:27.125136 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-4690b8a3-26aa-458f-824e-51addd4c38d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:27.139167 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4690b8a3-26aa-458f-824e-51addd4c38d2 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:27.166326 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:13:27.198425 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a21b8-7836-45dc-ac1c-5229df6c0df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:27.199086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a21b8-7836-45dc-ac1c-5229df6c0df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:27.199322 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a21b8-7836-45dc-ac1c-5229df6c0df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:27.199594 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a21b8-7836-45dc-ac1c-5229df6c0df2 admin admin] PATH_INFO: `/v3/projects/8f7a705a24cd41cf9fb1880c50c746b1` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:27.200779 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a21b8-7836-45dc-ac1c-5229df6c0df2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f7a705a24cd41cf9fb1880c50c746b1)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:27.202666 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a21b8-7836-45dc-ac1c-5229df6c0df2 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:27.250351 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-732a21b8-7836-45dc-ac1c-5229df6c0df2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f7a705a24cd41cf9fb1880c50c746b1', 'request_id': 'req-732a21b8-7836-45dc-ac1c-5229df6c0df2'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:27.255709 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-732a21b8-7836-45dc-ac1c-5229df6c0df2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f7a705a24cd41cf9fb1880c50c746b1', 'request_id': 'req-732a21b8-7836-45dc-ac1c-5229df6c0df2'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:27.262731 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-732a21b8-7836-45dc-ac1c-5229df6c0df2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f7a705a24cd41cf9fb1880c50c746b1', 'request_id': 'req-732a21b8-7836-45dc-ac1c-5229df6c0df2'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:27.274542 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-732a21b8-7836-45dc-ac1c-5229df6c0df2 admin admin] The token cache is being invalidate because project 8f7a705a24cd41cf9fb1880c50c746b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:13:27.274833 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-732a21b8-7836-45dc-ac1c-5229df6c0df2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:27.275463 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-732a21b8-7836-45dc-ac1c-5229df6c0df2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:27.277031 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1445/2926] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:13:27 2022] DELETE /identity/v3/projects/8f7a705a24cd41cf9fb1880c50c746b1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:13:27.282760 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94884d-6b18-4f38-b881-38443d23532d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:27.283231 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94884d-6b18-4f38-b881-38443d23532d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:27.283341 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94884d-6b18-4f38-b881-38443d23532d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:27.304215 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4a94884d-6b18-4f38-b881-38443d23532d None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:13:27.305459 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4a94884d-6b18-4f38-b881-38443d23532d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:27.363142 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1482/2927] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:13:27.371076 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da60e80-a5d0-43c6-9ca4-4a8d736a48b7 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:27.371359 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da60e80-a5d0-43c6-9ca4-4a8d736a48b7 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:27.371631 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da60e80-a5d0-43c6-9ca4-4a8d736a48b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:27.390971 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-7da60e80-a5d0-43c6-9ca4-4a8d736a48b7 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:13:27.392146 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7da60e80-a5d0-43c6-9ca4-4a8d736a48b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:27.400423 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1446/2928] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:13:27.404526 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94884d-6b18-4f38-b881-38443d23532d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:27.414848 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94884d-6b18-4f38-b881-38443d23532d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:27.420334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc417d-7b06-4172-aa8c-80b51141ca7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:27.420334 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc417d-7b06-4172-aa8c-80b51141ca7d admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:27.420585 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc417d-7b06-4172-aa8c-80b51141ca7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:27.420733 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc417d-7b06-4172-aa8c-80b51141ca7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:27.421454 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc417d-7b06-4172-aa8c-80b51141ca7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:13:27.421751 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc417d-7b06-4172-aa8c-80b51141ca7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:27.423331 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc417d-7b06-4172-aa8c-80b51141ca7d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:27.435969 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1483/2929] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 06 13:13:27.440616 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da60e80-a5d0-43c6-9ca4-4a8d736a48b7 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:27.452450 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da60e80-a5d0-43c6-9ca4-4a8d736a48b7 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:27.457660 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ad561-66f7-4032-92fe-567d4f0b6ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:27.458278 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ad561-66f7-4032-92fe-567d4f0b6ee4 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:27.458565 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ad561-66f7-4032-92fe-567d4f0b6ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:27.458843 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ad561-66f7-4032-92fe-567d4f0b6ee4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:27.459505 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ad561-66f7-4032-92fe-567d4f0b6ee4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:27.461444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ad561-66f7-4032-92fe-567d4f0b6ee4 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:27.493218 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1447/2930] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:13:27 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:13:27.497187 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc417d-7b06-4172-aa8c-80b51141ca7d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:27.508168 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc417d-7b06-4172-aa8c-80b51141ca7d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:27.564442 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132dc7e2-637f-4b1a-baf8-2e2ca5e84e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:27.565589 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132dc7e2-637f-4b1a-baf8-2e2ca5e84e5d admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:27.565866 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132dc7e2-637f-4b1a-baf8-2e2ca5e84e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:27.566138 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132dc7e2-637f-4b1a-baf8-2e2ca5e84e5d admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:27.566824 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132dc7e2-637f-4b1a-baf8-2e2ca5e84e5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:27.568761 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132dc7e2-637f-4b1a-baf8-2e2ca5e84e5d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:27.590069 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1484/2931] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:13:27 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:13:27.594341 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ad561-66f7-4032-92fe-567d4f0b6ee4 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:27.605429 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ad561-66f7-4032-92fe-567d4f0b6ee4 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:27.610119 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c7ecf-657c-4aa2-86b1-c1b089240cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:27.610753 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c7ecf-657c-4aa2-86b1-c1b089240cfa admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:27.611032 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c7ecf-657c-4aa2-86b1-c1b089240cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:27.611299 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c7ecf-657c-4aa2-86b1-c1b089240cfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:27.612056 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c7ecf-657c-4aa2-86b1-c1b089240cfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:27.613793 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c7ecf-657c-4aa2-86b1-c1b089240cfa admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:27.623607 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1448/2932] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 13:13:27.628092 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132dc7e2-637f-4b1a-baf8-2e2ca5e84e5d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:27.638119 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132dc7e2-637f-4b1a-baf8-2e2ca5e84e5d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:27.643596 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6133a7-3471-482c-8f39-f6394c514cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:27.644375 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6133a7-3471-482c-8f39-f6394c514cbd admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:27.644658 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6133a7-3471-482c-8f39-f6394c514cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:27.644933 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6133a7-3471-482c-8f39-f6394c514cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:27.645758 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6133a7-3471-482c-8f39-f6394c514cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:27.647411 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6133a7-3471-482c-8f39-f6394c514cbd admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:27.660729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1485/2933] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 06 13:13:27.664857 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c7ecf-657c-4aa2-86b1-c1b089240cfa admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:27.675164 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c7ecf-657c-4aa2-86b1-c1b089240cfa admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:27.680544 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdaf044-2e09-4ce0-a9e0-f723b794bcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:27.681207 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdaf044-2e09-4ce0-a9e0-f723b794bcee admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:27.681486 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdaf044-2e09-4ce0-a9e0-f723b794bcee admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:27.681759 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdaf044-2e09-4ce0-a9e0-f723b794bcee admin admin] PATH_INFO: `/v3/projects/a716b7e2d3024c70bb85f6e2f2dfdb72/users/afdfeddca46844099f36ff713e65f2ab/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:27.696745 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdaf044-2e09-4ce0-a9e0-f723b794bcee admin admin] RBAC: Authorizing `identity:create_grant(project_id=a716b7e2d3024c70bb85f6e2f2dfdb72, user_id=afdfeddca46844099f36ff713e65f2ab, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:27.699364 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdaf044-2e09-4ce0-a9e0-f723b794bcee admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:27.712461 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1449/2934] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:13:27 2022] PUT /identity/v3/projects/a716b7e2d3024c70bb85f6e2f2dfdb72/users/afdfeddca46844099f36ff713e65f2ab/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 06 13:13:27.723788 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3722a7-aa9e-49fc-92fa-69ad4848be76 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:27.724075 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3722a7-aa9e-49fc-92fa-69ad4848be76 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:27.724388 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3722a7-aa9e-49fc-92fa-69ad4848be76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:27.751251 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-fd3722a7-aa9e-49fc-92fa-69ad4848be76 None None] MFA Rules not processed for user `afdfeddca46844099f36ff713e65f2ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:13:27.753053 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fd3722a7-aa9e-49fc-92fa-69ad4848be76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:27.799311 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:13:27.817206 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1486/2935] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:13:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:13:27.829007 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdaf044-2e09-4ce0-a9e0-f723b794bcee admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:27.831315 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6fdaf044-2e09-4ce0-a9e0-f723b794bcee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:27.885900 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdaf044-2e09-4ce0-a9e0-f723b794bcee admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:27.889392 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd278fb-8df1-4fd0-aceb-98ad943860ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7a3cbb9c6a34fe2b280873771532e4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:27.889992 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd278fb-8df1-4fd0-aceb-98ad943860ae service cinder] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:27.890267 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd278fb-8df1-4fd0-aceb-98ad943860ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:27.890535 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd278fb-8df1-4fd0-aceb-98ad943860ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:27.898651 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd278fb-8df1-4fd0-aceb-98ad943860ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:27.900060 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd278fb-8df1-4fd0-aceb-98ad943860ae service cinder] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:27.903477 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:13:27.910269 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1450/2936] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:13:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:13:27.931737 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3722a7-aa9e-49fc-92fa-69ad4848be76 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:27.994491 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3722a7-aa9e-49fc-92fa-69ad4848be76 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:27.999219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf1e43-cc47-4263-8b07-cc6f69a39c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:27.999850 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf1e43-cc47-4263-8b07-cc6f69a39c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:28.000126 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf1e43-cc47-4263-8b07-cc6f69a39c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:28.000426 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf1e43-cc47-4263-8b07-cc6f69a39c38 admin admin] PATH_INFO: `/v3/users/afdfeddca46844099f36ff713e65f2ab` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:28.001601 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf1e43-cc47-4263-8b07-cc6f69a39c38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afdfeddca46844099f36ff713e65f2ab)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:28.004113 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf1e43-cc47-4263-8b07-cc6f69a39c38 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:28.048447 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-51bf1e43-cc47-4263-8b07-cc6f69a39c38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afdfeddca46844099f36ff713e65f2ab'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:28.060758 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-51bf1e43-cc47-4263-8b07-cc6f69a39c38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afdfeddca46844099f36ff713e65f2ab'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:28.061428 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-51bf1e43-cc47-4263-8b07-cc6f69a39c38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afdfeddca46844099f36ff713e65f2ab'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:28.062062 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-51bf1e43-cc47-4263-8b07-cc6f69a39c38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afdfeddca46844099f36ff713e65f2ab'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:28.069745 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-51bf1e43-cc47-4263-8b07-cc6f69a39c38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afdfeddca46844099f36ff713e65f2ab'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:28.081759 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1487/2937] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:13:27 2022] DELETE /identity/v3/users/afdfeddca46844099f36ff713e65f2ab => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:13:28.086257 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f48f6-b368-46c5-94d4-d71e99a56229 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:28.086544 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f48f6-b368-46c5-94d4-d71e99a56229 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:28.086843 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f48f6-b368-46c5-94d4-d71e99a56229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:28.105528 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-2c9f48f6-b368-46c5-94d4-d71e99a56229 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:13:28.106746 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2c9f48f6-b368-46c5-94d4-d71e99a56229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:28.164327 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1451/2938] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:13:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:13:28.173937 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf1e43-cc47-4263-8b07-cc6f69a39c38 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:28.175867 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-51bf1e43-cc47-4263-8b07-cc6f69a39c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:28.232508 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf1e43-cc47-4263-8b07-cc6f69a39c38 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:28.236136 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623bc218-b2be-4460-bb06-cec7f875a66f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:28.237419 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bc218-b2be-4460-bb06-cec7f875a66f service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:28.237704 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bc218-b2be-4460-bb06-cec7f875a66f service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:28.237980 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bc218-b2be-4460-bb06-cec7f875a66f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:28.246889 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623bc218-b2be-4460-bb06-cec7f875a66f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:28.248330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623bc218-b2be-4460-bb06-cec7f875a66f service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:28.257785 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1488/2939] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:13:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:13:28.524806 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f48f6-b368-46c5-94d4-d71e99a56229 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:28.526278 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2c9f48f6-b368-46c5-94d4-d71e99a56229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:28.541770 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f48f6-b368-46c5-94d4-d71e99a56229 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:28.547116 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53884b9-9600-408b-afdd-97eb1f5dfd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:28.547758 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53884b9-9600-408b-afdd-97eb1f5dfd80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:28.548086 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53884b9-9600-408b-afdd-97eb1f5dfd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:28.548531 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53884b9-9600-408b-afdd-97eb1f5dfd80 admin admin] PATH_INFO: `/v3/projects/a716b7e2d3024c70bb85f6e2f2dfdb72` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:28.550337 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53884b9-9600-408b-afdd-97eb1f5dfd80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a716b7e2d3024c70bb85f6e2f2dfdb72)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:28.552279 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53884b9-9600-408b-afdd-97eb1f5dfd80 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:28.602841 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f53884b9-9600-408b-afdd-97eb1f5dfd80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a716b7e2d3024c70bb85f6e2f2dfdb72', 'request_id': 'req-f53884b9-9600-408b-afdd-97eb1f5dfd80'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:28.608542 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f53884b9-9600-408b-afdd-97eb1f5dfd80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a716b7e2d3024c70bb85f6e2f2dfdb72', 'request_id': 'req-f53884b9-9600-408b-afdd-97eb1f5dfd80'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:28.615600 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f53884b9-9600-408b-afdd-97eb1f5dfd80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a716b7e2d3024c70bb85f6e2f2dfdb72', 'request_id': 'req-f53884b9-9600-408b-afdd-97eb1f5dfd80'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:28.626320 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f53884b9-9600-408b-afdd-97eb1f5dfd80 admin admin] The token cache is being invalidate because project a716b7e2d3024c70bb85f6e2f2dfdb72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:13:28.626605 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f53884b9-9600-408b-afdd-97eb1f5dfd80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:28.627247 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-f53884b9-9600-408b-afdd-97eb1f5dfd80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:13:28.628775 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1452/2940] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:13:28 2022] DELETE /identity/v3/projects/a716b7e2d3024c70bb85f6e2f2dfdb72 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:13:28.633615 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea718c-30a1-43b1-a486-ea69ae3d38bb None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:28.634104 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea718c-30a1-43b1-a486-ea69ae3d38bb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:28.634234 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea718c-30a1-43b1-a486-ea69ae3d38bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:28.653708 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-07ea718c-30a1-43b1-a486-ea69ae3d38bb None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:13:28.654907 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-07ea718c-30a1-43b1-a486-ea69ae3d38bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:28.663828 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:13:28.715138 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1489/2941] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:13:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:13:28.724645 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804969e8-870c-4d12-a2d6-c5e8df3c7b89 None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:28.724928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804969e8-870c-4d12-a2d6-c5e8df3c7b89 None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:28.725198 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804969e8-870c-4d12-a2d6-c5e8df3c7b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:28.744220 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-804969e8-870c-4d12-a2d6-c5e8df3c7b89 None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:13:28.745427 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-804969e8-870c-4d12-a2d6-c5e8df3c7b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:28.753207 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1453/2942] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:13:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:13:28.757219 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea718c-30a1-43b1-a486-ea69ae3d38bb None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:28.766668 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea718c-30a1-43b1-a486-ea69ae3d38bb None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:28.771694 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2093ded-5247-4e8f-83e0-a0003c3b715b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:28.772289 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2093ded-5247-4e8f-83e0-a0003c3b715b admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:28.772595 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2093ded-5247-4e8f-83e0-a0003c3b715b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:28.772866 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2093ded-5247-4e8f-83e0-a0003c3b715b admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:28.773550 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2093ded-5247-4e8f-83e0-a0003c3b715b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 06 13:13:28.773849 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2093ded-5247-4e8f-83e0-a0003c3b715b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:28.775415 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2093ded-5247-4e8f-83e0-a0003c3b715b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:28.787269 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1490/2943] 10.222.0.20 () {58 vars in 1153 bytes} [Sat Aug 6 13:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 06 13:13:28.791206 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804969e8-870c-4d12-a2d6-c5e8df3c7b89 None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:28.800333 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804969e8-870c-4d12-a2d6-c5e8df3c7b89 None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:28.804884 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb14ab2-adbe-4ff1-942c-ced8c8b6d687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:28.805476 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb14ab2-adbe-4ff1-942c-ced8c8b6d687 admin admin] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:28.805771 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb14ab2-adbe-4ff1-942c-ced8c8b6d687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:28.806042 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb14ab2-adbe-4ff1-942c-ced8c8b6d687 admin admin] PATH_INFO: `/v3/projects` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:28.806698 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb14ab2-adbe-4ff1-942c-ced8c8b6d687 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:28.808604 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb14ab2-adbe-4ff1-942c-ced8c8b6d687 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:28.835523 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1454/2944] 10.222.0.20 () {60 vars in 1153 bytes} [Sat Aug 6 13:13:28 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 06 13:13:28.839505 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2093ded-5247-4e8f-83e0-a0003c3b715b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:28.850258 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2093ded-5247-4e8f-83e0-a0003c3b715b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:28.904148 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc954b9-02df-41d7-97c6-2539f91de10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:28.904784 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc954b9-02df-41d7-97c6-2539f91de10a admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:28.905062 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc954b9-02df-41d7-97c6-2539f91de10a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:28.905330 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc954b9-02df-41d7-97c6-2539f91de10a admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:28.905972 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc954b9-02df-41d7-97c6-2539f91de10a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:28.907852 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc954b9-02df-41d7-97c6-2539f91de10a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:28.928682 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1491/2945] 10.222.0.20 () {60 vars in 1144 bytes} [Sat Aug 6 13:13:28 2022] POST /identity/v3/users => generated 284 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 06 13:13:28.932985 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb14ab2-adbe-4ff1-942c-ced8c8b6d687 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:28.943832 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb14ab2-adbe-4ff1-942c-ced8c8b6d687 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:28.948411 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ddbd15-36ca-4ee4-8406-22048b263602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:28.949006 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddbd15-36ca-4ee4-8406-22048b263602 admin admin] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:28.949283 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddbd15-36ca-4ee4-8406-22048b263602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:28.949554 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddbd15-36ca-4ee4-8406-22048b263602 admin admin] PATH_INFO: `/v3/roles` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:28.950284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ddbd15-36ca-4ee4-8406-22048b263602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:28.951966 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ddbd15-36ca-4ee4-8406-22048b263602 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:28.962220 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1455/2946] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:13:28.966202 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc954b9-02df-41d7-97c6-2539f91de10a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:28.977503 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc954b9-02df-41d7-97c6-2539f91de10a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:28.982152 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c9ffc-f31b-414f-8915-926f11cfacb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:28.982754 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c9ffc-f31b-414f-8915-926f11cfacb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:28.983037 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c9ffc-f31b-414f-8915-926f11cfacb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:28.983312 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c9ffc-f31b-414f-8915-926f11cfacb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:28.984059 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c9ffc-f31b-414f-8915-926f11cfacb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:28.985687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c9ffc-f31b-414f-8915-926f11cfacb6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:28.998685 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1492/2947] 10.222.0.20 () {58 vars in 1122 bytes} [Sat Aug 6 13:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 06 13:13:29.003444 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ddbd15-36ca-4ee4-8406-22048b263602 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:29.013665 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ddbd15-36ca-4ee4-8406-22048b263602 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:29.019680 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5cdaa-3978-4fc1-8942-269ce9b49d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:29.020372 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5cdaa-3978-4fc1-8942-269ce9b49d91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:29.020649 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5cdaa-3978-4fc1-8942-269ce9b49d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:29.020917 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5cdaa-3978-4fc1-8942-269ce9b49d91 admin admin] PATH_INFO: `/v3/projects/559c860b83c645ed9c8f76b57e230c0f/users/c2ea4d7414ca482ab576cac05abc3990/roles/0fd7b75299b64c54971d9bfeaec30f4e` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:29.035268 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5cdaa-3978-4fc1-8942-269ce9b49d91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=559c860b83c645ed9c8f76b57e230c0f, user_id=c2ea4d7414ca482ab576cac05abc3990, role_id=0fd7b75299b64c54971d9bfeaec30f4e)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:29.037915 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5cdaa-3978-4fc1-8942-269ce9b49d91 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:29.050310 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1456/2948] 10.222.0.20 () {60 vars in 1483 bytes} [Sat Aug 6 13:13:29 2022] PUT /identity/v3/projects/559c860b83c645ed9c8f76b57e230c0f/users/c2ea4d7414ca482ab576cac05abc3990/roles/0fd7b75299b64c54971d9bfeaec30f4e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:13:29.061856 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87734a57-9082-48ff-a870-fc896ce54077 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:29.062149 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87734a57-9082-48ff-a870-fc896ce54077 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:29.062447 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87734a57-9082-48ff-a870-fc896ce54077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:29.091305 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-87734a57-9082-48ff-a870-fc896ce54077 None None] MFA Rules not processed for user `c2ea4d7414ca482ab576cac05abc3990`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:13:29.092543 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-87734a57-9082-48ff-a870-fc896ce54077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:29.155033 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1493/2949] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:13:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:13:29.167284 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5cdaa-3978-4fc1-8942-269ce9b49d91 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:13:29.169014 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f3a5cdaa-3978-4fc1-8942-269ce9b49d91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:13:29.188519 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:13:29.226584 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5cdaa-3978-4fc1-8942-269ce9b49d91 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:13:29.230172 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffabb24-e660-41f7-b04e-3d4660b11d97 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7610d6289efb44599d0cee03b93f9938', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:13:29.230776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffabb24-e660-41f7-b04e-3d4660b11d97 service glance] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:13:29.231057 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffabb24-e660-41f7-b04e-3d4660b11d97 service glance] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:13:29.231332 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffabb24-e660-41f7-b04e-3d4660b11d97 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:13:29.239477 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffabb24-e660-41f7-b04e-3d4660b11d97 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:13:29.240940 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffabb24-e660-41f7-b04e-3d4660b11d97 service glance] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:13:29.250211 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1457/2950] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:13:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:13:39.152089 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=93,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:13:39.245645 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=18 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:14:09.855752 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87734a57-9082-48ff-a870-fc896ce54077 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:14:09.916386 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87734a57-9082-48ff-a870-fc896ce54077 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:14:09.920807 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e563c4c2-dba4-46fc-be32-df33df06d266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:14:09.921435 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563c4c2-dba4-46fc-be32-df33df06d266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:14:09.921714 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563c4c2-dba4-46fc-be32-df33df06d266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:14:09.921984 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563c4c2-dba4-46fc-be32-df33df06d266 admin admin] PATH_INFO: `/v3/users/c2ea4d7414ca482ab576cac05abc3990` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:14:09.923120 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e563c4c2-dba4-46fc-be32-df33df06d266 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2ea4d7414ca482ab576cac05abc3990)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:14:09.925056 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e563c4c2-dba4-46fc-be32-df33df06d266 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:14:09.970635 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e563c4c2-dba4-46fc-be32-df33df06d266 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2ea4d7414ca482ab576cac05abc3990'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:14:09.982820 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e563c4c2-dba4-46fc-be32-df33df06d266 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2ea4d7414ca482ab576cac05abc3990'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:14:09.983494 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e563c4c2-dba4-46fc-be32-df33df06d266 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2ea4d7414ca482ab576cac05abc3990'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:14:09.984221 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e563c4c2-dba4-46fc-be32-df33df06d266 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2ea4d7414ca482ab576cac05abc3990'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:14:09.991584 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e563c4c2-dba4-46fc-be32-df33df06d266 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2ea4d7414ca482ab576cac05abc3990'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:14:10.004779 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1494/2951] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:14:09 2022] DELETE /identity/v3/users/c2ea4d7414ca482ab576cac05abc3990 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:14:10.009834 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bda751-4bfe-496e-84e4-03597079030f None None] REQUEST_METHOD: `POST` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:14:10.010280 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bda751-4bfe-496e-84e4-03597079030f None None] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:14:10.010382 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bda751-4bfe-496e-84e4-03597079030f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:14:10.029812 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.auth.core [None req-f0bda751-4bfe-496e-84e4-03597079030f None None] MFA Rules not processed for user `c9de148de73e459eb6456afdaf9fe213`. Rule list: `[]` (Enabled: `True`). {{(pid=80145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 06 13:14:10.031002 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f0bda751-4bfe-496e-84e4-03597079030f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:14:10.086939 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1458/2952] 10.222.0.20 () {58 vars in 958 bytes} [Sat Aug 6 13:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 06 13:14:10.097276 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e563c4c2-dba4-46fc-be32-df33df06d266 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:14:10.098646 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e563c4c2-dba4-46fc-be32-df33df06d266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:14:10.153687 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e563c4c2-dba4-46fc-be32-df33df06d266 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:14:10.157283 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40465c-c787-4f89-9d18-5dfaca69a9aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:14:10.157877 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40465c-c787-4f89-9d18-5dfaca69a9aa service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:14:10.158155 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40465c-c787-4f89-9d18-5dfaca69a9aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:14:10.158427 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40465c-c787-4f89-9d18-5dfaca69a9aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:14:10.167225 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40465c-c787-4f89-9d18-5dfaca69a9aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:14:10.168663 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40465c-c787-4f89-9d18-5dfaca69a9aa service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:14:10.177854 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1495/2953] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:14:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:14:10.460995 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bda751-4bfe-496e-84e4-03597079030f None None] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:14:10.462443 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-f0bda751-4bfe-496e-84e4-03597079030f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:14:10.476378 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bda751-4bfe-496e-84e4-03597079030f None None] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:14:10.481057 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6ef48-be31-47d9-93aa-e57833c29626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:14:10.481700 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6ef48-be31-47d9-93aa-e57833c29626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:14:10.481980 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6ef48-be31-47d9-93aa-e57833c29626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:14:10.482252 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6ef48-be31-47d9-93aa-e57833c29626 admin admin] PATH_INFO: `/v3/projects/559c860b83c645ed9c8f76b57e230c0f` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:14:10.483403 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6ef48-be31-47d9-93aa-e57833c29626 admin admin] RBAC: Authorizing `identity:delete_project(project_id=559c860b83c645ed9c8f76b57e230c0f)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:14:10.486433 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6ef48-be31-47d9-93aa-e57833c29626 admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:14:10.534458 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-2ce6ef48-be31-47d9-93aa-e57833c29626 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '559c860b83c645ed9c8f76b57e230c0f', 'request_id': 'req-2ce6ef48-be31-47d9-93aa-e57833c29626'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:14:10.539928 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-2ce6ef48-be31-47d9-93aa-e57833c29626 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '559c860b83c645ed9c8f76b57e230c0f', 'request_id': 'req-2ce6ef48-be31-47d9-93aa-e57833c29626'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:14:10.546791 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-2ce6ef48-be31-47d9-93aa-e57833c29626 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '559c860b83c645ed9c8f76b57e230c0f', 'request_id': 'req-2ce6ef48-be31-47d9-93aa-e57833c29626'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:14:10.558406 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-2ce6ef48-be31-47d9-93aa-e57833c29626 admin admin] The token cache is being invalidate because project 559c860b83c645ed9c8f76b57e230c0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:14:10.558692 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-2ce6ef48-be31-47d9-93aa-e57833c29626 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:14:10.559312 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-2ce6ef48-be31-47d9-93aa-e57833c29626 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:14:10.560868 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1459/2954] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:14:10 2022] DELETE /identity/v3/projects/559c860b83c645ed9c8f76b57e230c0f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:14:20.173810 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:14:20.554924 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:14:23.179740 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40465c-c787-4f89-9d18-5dfaca69a9aa service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:14:23.188493 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40465c-c787-4f89-9d18-5dfaca69a9aa service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:14:23.236729 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1236cd4-9c46-44df-b48a-0c5392fa3d02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:14:23.237335 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1236cd4-9c46-44df-b48a-0c5392fa3d02 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:14:23.237614 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1236cd4-9c46-44df-b48a-0c5392fa3d02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:14:23.237886 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1236cd4-9c46-44df-b48a-0c5392fa3d02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:14:23.239642 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f1236cd4-9c46-44df-b48a-0c5392fa3d02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:14:23.314783 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1236cd4-9c46-44df-b48a-0c5392fa3d02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:14:23.316207 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1236cd4-9c46-44df-b48a-0c5392fa3d02 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:14:23.327116 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1496/2955] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:14:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:14:24.505066 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6ef48-be31-47d9-93aa-e57833c29626 admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:14:24.506467 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-2ce6ef48-be31-47d9-93aa-e57833c29626 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:14:24.519318 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6ef48-be31-47d9-93aa-e57833c29626 admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:14:24.522850 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf5f939-a7a3-427d-baf2-a1f30f8d99bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:14:24.523447 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf5f939-a7a3-427d-baf2-a1f30f8d99bf service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:14:24.523723 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf5f939-a7a3-427d-baf2-a1f30f8d99bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:14:24.523994 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf5f939-a7a3-427d-baf2-a1f30f8d99bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:14:24.525706 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-6cf5f939-a7a3-427d-baf2-a1f30f8d99bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:14:24.538585 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf5f939-a7a3-427d-baf2-a1f30f8d99bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:14:24.539990 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf5f939-a7a3-427d-baf2-a1f30f8d99bf service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:14:24.550715 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1460/2956] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:14:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:14:33.322177 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=54 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:14:34.545896 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=6 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:14:36.526669 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1236cd4-9c46-44df-b48a-0c5392fa3d02 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:14:36.528101 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f1236cd4-9c46-44df-b48a-0c5392fa3d02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:14:36.538777 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1236cd4-9c46-44df-b48a-0c5392fa3d02 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:14:36.542617 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b15f0-f924-4e7f-bb0e-eed64ed95e07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:14:36.543235 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b15f0-f924-4e7f-bb0e-eed64ed95e07 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:14:36.543519 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b15f0-f924-4e7f-bb0e-eed64ed95e07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:14:36.543797 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b15f0-f924-4e7f-bb0e-eed64ed95e07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:14:36.545549 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-003b15f0-f924-4e7f-bb0e-eed64ed95e07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:14:36.555709 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b15f0-f924-4e7f-bb0e-eed64ed95e07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:14:36.557699 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b15f0-f924-4e7f-bb0e-eed64ed95e07 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:14:36.566431 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1497/2957] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:14:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:14:36.645714 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf5f939-a7a3-427d-baf2-a1f30f8d99bf service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:14:36.654769 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf5f939-a7a3-427d-baf2-a1f30f8d99bf service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:14:36.658385 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a0e77-5853-443b-953b-d59ebfdd07be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:14:36.658989 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a0e77-5853-443b-953b-d59ebfdd07be service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:14:36.672195 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a0e77-5853-443b-953b-d59ebfdd07be service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:14:36.672195 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a0e77-5853-443b-953b-d59ebfdd07be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:14:36.672195 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-7e3a0e77-5853-443b-953b-d59ebfdd07be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:14:36.675164 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a0e77-5853-443b-953b-d59ebfdd07be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:14:36.676603 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a0e77-5853-443b-953b-d59ebfdd07be service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:14:36.687837 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1461/2958] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:14:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:14:36.747465 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b15f0-f924-4e7f-bb0e-eed64ed95e07 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:14:36.748890 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-003b15f0-f924-4e7f-bb0e-eed64ed95e07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:14:36.817795 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b15f0-f924-4e7f-bb0e-eed64ed95e07 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:14:36.822782 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890375a-8c1a-4ecd-9e07-5e96107f07e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2244655ba304b9c980b5e68772eefc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:14:36.823386 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890375a-8c1a-4ecd-9e07-5e96107f07e0 service placement] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:14:36.823660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890375a-8c1a-4ecd-9e07-5e96107f07e0 service placement] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:14:36.823933 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890375a-8c1a-4ecd-9e07-5e96107f07e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:14:36.825680 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6890375a-8c1a-4ecd-9e07-5e96107f07e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:14:36.837123 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890375a-8c1a-4ecd-9e07-5e96107f07e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:14:36.838715 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890375a-8c1a-4ecd-9e07-5e96107f07e0 service placement] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:14:36.848744 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1498/2959] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:14:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:14:46.683106 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=6 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:14:46.844169 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=38 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:15:06.010481 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a0e77-5853-443b-953b-d59ebfdd07be service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:15:06.021646 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a0e77-5853-443b-953b-d59ebfdd07be service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:15:06.026522 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd90bcc-cf39-4b63-942b-6a9a69c7dda3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2244655ba304b9c980b5e68772eefc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:15:06.027115 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd90bcc-cf39-4b63-942b-6a9a69c7dda3 service placement] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:15:06.027389 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd90bcc-cf39-4b63-942b-6a9a69c7dda3 service placement] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:15:06.027657 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd90bcc-cf39-4b63-942b-6a9a69c7dda3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:15:06.029458 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-0cd90bcc-cf39-4b63-942b-6a9a69c7dda3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:15:06.053133 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd90bcc-cf39-4b63-942b-6a9a69c7dda3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:15:06.054696 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd90bcc-cf39-4b63-942b-6a9a69c7dda3 service placement] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:15:06.065064 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1462/2960] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:15:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:15:16.060734 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=11 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:16:24.687798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890375a-8c1a-4ecd-9e07-5e96107f07e0 service placement] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:16:24.697129 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890375a-8c1a-4ecd-9e07-5e96107f07e0 service placement] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:16:24.701036 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0645192-fb7a-4953-942b-3d8d8e52e441 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:16:24.701627 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0645192-fb7a-4953-942b-3d8d8e52e441 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:16:24.701903 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0645192-fb7a-4953-942b-3d8d8e52e441 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:16:24.702172 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0645192-fb7a-4953-942b-3d8d8e52e441 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:16:24.703867 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c0645192-fb7a-4953-942b-3d8d8e52e441 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:16:24.760493 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0645192-fb7a-4953-942b-3d8d8e52e441 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:16:24.761904 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0645192-fb7a-4953-942b-3d8d8e52e441 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:16:24.770556 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1499/2961] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:16:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:16:29.459610 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd90bcc-cf39-4b63-942b-6a9a69c7dda3 service placement] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:16:29.469879 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd90bcc-cf39-4b63-942b-6a9a69c7dda3 service placement] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:16:29.473573 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9945d-cdb9-4f66-9aa1-a84d4055d1e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55563aaa662246c5aaf107145475bd76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:16:29.474183 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9945d-cdb9-4f66-9aa1-a84d4055d1e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:16:29.474468 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9945d-cdb9-4f66-9aa1-a84d4055d1e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:16:29.475012 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9945d-cdb9-4f66-9aa1-a84d4055d1e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:16:29.476817 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-63b9945d-cdb9-4f66-9aa1-a84d4055d1e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:16:29.564210 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b9945d-cdb9-4f66-9aa1-a84d4055d1e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:16:29.565674 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b9945d-cdb9-4f66-9aa1-a84d4055d1e6 service neutron] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:16:29.584598 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1463/2962] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:16:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:16:32.611369 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0645192-fb7a-4953-942b-3d8d8e52e441 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:16:32.612947 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c0645192-fb7a-4953-942b-3d8d8e52e441 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:16:32.629480 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0645192-fb7a-4953-942b-3d8d8e52e441 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:16:32.639468 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1c78b-d99a-43ae-bacb-54ae527fd99e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2244655ba304b9c980b5e68772eefc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:16:32.640096 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1c78b-d99a-43ae-bacb-54ae527fd99e service placement] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:16:32.640413 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1c78b-d99a-43ae-bacb-54ae527fd99e service placement] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:16:32.640695 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1c78b-d99a-43ae-bacb-54ae527fd99e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:16:32.642557 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fab1c78b-d99a-43ae-bacb-54ae527fd99e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:16:32.656380 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1c78b-d99a-43ae-bacb-54ae527fd99e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:16:32.657996 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1c78b-d99a-43ae-bacb-54ae527fd99e service placement] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:16:32.668893 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1500/2963] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:16:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:16:35.750199 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9945d-cdb9-4f66-9aa1-a84d4055d1e6 service neutron] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:16:35.752421 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-63b9945d-cdb9-4f66-9aa1-a84d4055d1e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:16:35.766575 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9945d-cdb9-4f66-9aa1-a84d4055d1e6 service neutron] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:16:35.772469 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc02cfe-488f-42cb-8b5b-715c16fd813b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:16:35.773575 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc02cfe-488f-42cb-8b5b-715c16fd813b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:16:35.774085 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc02cfe-488f-42cb-8b5b-715c16fd813b admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:16:35.774607 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc02cfe-488f-42cb-8b5b-715c16fd813b admin admin] PATH_INFO: `/v3/users/2dacef0fd30a43d2b29a2eeb67884ec1` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:16:35.775788 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc02cfe-488f-42cb-8b5b-715c16fd813b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dacef0fd30a43d2b29a2eeb67884ec1)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:16:35.777747 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc02cfe-488f-42cb-8b5b-715c16fd813b admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:16:35.824974 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-1fc02cfe-488f-42cb-8b5b-715c16fd813b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dacef0fd30a43d2b29a2eeb67884ec1'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:16:35.832182 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-1fc02cfe-488f-42cb-8b5b-715c16fd813b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dacef0fd30a43d2b29a2eeb67884ec1'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:16:35.844493 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-1fc02cfe-488f-42cb-8b5b-715c16fd813b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dacef0fd30a43d2b29a2eeb67884ec1'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:16:35.845162 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-1fc02cfe-488f-42cb-8b5b-715c16fd813b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dacef0fd30a43d2b29a2eeb67884ec1'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:16:35.845772 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-1fc02cfe-488f-42cb-8b5b-715c16fd813b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dacef0fd30a43d2b29a2eeb67884ec1'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:16:35.858787 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1464/2964] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:16:35 2022] DELETE /identity/v3/users/2dacef0fd30a43d2b29a2eeb67884ec1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:16:36.034088 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1c78b-d99a-43ae-bacb-54ae527fd99e service placement] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:16:36.047285 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1c78b-d99a-43ae-bacb-54ae527fd99e service placement] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:16:36.111660 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d85334-a91e-40b9-8719-19a544364f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:16:36.112326 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d85334-a91e-40b9-8719-19a544364f06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:16:36.112623 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d85334-a91e-40b9-8719-19a544364f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:16:36.112895 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d85334-a91e-40b9-8719-19a544364f06 admin admin] PATH_INFO: `/v3/projects/06e0dc4c7d244821bd974b0dea3fc631` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:16:36.114072 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d85334-a91e-40b9-8719-19a544364f06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06e0dc4c7d244821bd974b0dea3fc631)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:16:36.116063 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d85334-a91e-40b9-8719-19a544364f06 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:16:36.167332 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-33d85334-a91e-40b9-8719-19a544364f06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06e0dc4c7d244821bd974b0dea3fc631', 'request_id': 'req-33d85334-a91e-40b9-8719-19a544364f06'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:16:36.173798 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-33d85334-a91e-40b9-8719-19a544364f06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06e0dc4c7d244821bd974b0dea3fc631', 'request_id': 'req-33d85334-a91e-40b9-8719-19a544364f06'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:16:36.182290 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-33d85334-a91e-40b9-8719-19a544364f06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06e0dc4c7d244821bd974b0dea3fc631', 'request_id': 'req-33d85334-a91e-40b9-8719-19a544364f06'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:16:36.195158 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-33d85334-a91e-40b9-8719-19a544364f06 admin admin] The token cache is being invalidate because project 06e0dc4c7d244821bd974b0dea3fc631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:16:36.195452 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-33d85334-a91e-40b9-8719-19a544364f06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:16:36.196125 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-33d85334-a91e-40b9-8719-19a544364f06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:16:36.197703 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1501/2965] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:16:36 2022] DELETE /identity/v3/projects/06e0dc4c7d244821bd974b0dea3fc631 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:16:36.202373 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc02cfe-488f-42cb-8b5b-715c16fd813b admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:16:36.203852 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-1fc02cfe-488f-42cb-8b5b-715c16fd813b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:16:36.218420 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc02cfe-488f-42cb-8b5b-715c16fd813b admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:16:36.253926 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:16:36.280487 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08c664-c92e-468d-ae79-942138e72ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:16:36.281122 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08c664-c92e-468d-ae79-942138e72ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:16:36.281397 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08c664-c92e-468d-ae79-942138e72ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:16:36.281776 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08c664-c92e-468d-ae79-942138e72ecd admin admin] PATH_INFO: `/v3/users/2733d80bc47b432abf7adede6ef1cbb0` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:16:36.283729 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08c664-c92e-468d-ae79-942138e72ecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2733d80bc47b432abf7adede6ef1cbb0)` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:16:36.285685 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08c664-c92e-468d-ae79-942138e72ecd admin admin] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:16:36.332482 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3b08c664-c92e-468d-ae79-942138e72ecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2733d80bc47b432abf7adede6ef1cbb0'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:16:36.339118 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3b08c664-c92e-468d-ae79-942138e72ecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2733d80bc47b432abf7adede6ef1cbb0'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:16:36.351246 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3b08c664-c92e-468d-ae79-942138e72ecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2733d80bc47b432abf7adede6ef1cbb0'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:16:36.351926 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3b08c664-c92e-468d-ae79-942138e72ecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2733d80bc47b432abf7adede6ef1cbb0'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:16:36.352563 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.notifications [None req-3b08c664-c92e-468d-ae79-942138e72ecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2733d80bc47b432abf7adede6ef1cbb0'} {{(pid=80145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:16:36.365229 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1465/2966] 10.222.0.20 () {58 vars in 1224 bytes} [Sat Aug 6 13:16:36 2022] DELETE /identity/v3/users/2733d80bc47b432abf7adede6ef1cbb0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:16:36.531563 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d85334-a91e-40b9-8719-19a544364f06 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:16:36.533263 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-33d85334-a91e-40b9-8719-19a544364f06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:16:36.546695 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d85334-a91e-40b9-8719-19a544364f06 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:16:36.558894 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:16:36.606923 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a36b5-73b9-4fad-a59b-8f677eb3c5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9de148de73e459eb6456afdaf9fe213', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7ad154ab2bd64d7e8e712fcec9e1c94f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:16:36.608187 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a36b5-73b9-4fad-a59b-8f677eb3c5ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:16:36.608495 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a36b5-73b9-4fad-a59b-8f677eb3c5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:16:36.608767 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a36b5-73b9-4fad-a59b-8f677eb3c5ad admin admin] PATH_INFO: `/v3/projects/d7cfbcb88aa34ad1bb98a5dbfd849cb3` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:16:36.609915 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a36b5-73b9-4fad-a59b-8f677eb3c5ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7cfbcb88aa34ad1bb98a5dbfd849cb3)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:16:36.611804 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a36b5-73b9-4fad-a59b-8f677eb3c5ad admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:16:36.656190 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-aa7a36b5-73b9-4fad-a59b-8f677eb3c5ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7cfbcb88aa34ad1bb98a5dbfd849cb3', 'request_id': 'req-aa7a36b5-73b9-4fad-a59b-8f677eb3c5ad'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:16:36.661548 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-aa7a36b5-73b9-4fad-a59b-8f677eb3c5ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7cfbcb88aa34ad1bb98a5dbfd849cb3', 'request_id': 'req-aa7a36b5-73b9-4fad-a59b-8f677eb3c5ad'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:16:36.668674 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-aa7a36b5-73b9-4fad-a59b-8f677eb3c5ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7cfbcb88aa34ad1bb98a5dbfd849cb3', 'request_id': 'req-aa7a36b5-73b9-4fad-a59b-8f677eb3c5ad'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:16:36.679911 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-aa7a36b5-73b9-4fad-a59b-8f677eb3c5ad admin admin] The token cache is being invalidate because project d7cfbcb88aa34ad1bb98a5dbfd849cb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 06 13:16:36.680194 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-aa7a36b5-73b9-4fad-a59b-8f677eb3c5ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:16:36.680830 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-aa7a36b5-73b9-4fad-a59b-8f677eb3c5ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 06 13:16:36.682296 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1502/2967] 10.222.0.20 () {58 vars in 1233 bytes} [Sat Aug 6 13:16:36 2022] DELETE /identity/v3/projects/d7cfbcb88aa34ad1bb98a5dbfd849cb3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 06 13:16:46.361792 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:16:46.677102 dv-cinder-851996-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 06 13:20:11.018887 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08c664-c92e-468d-ae79-942138e72ecd admin admin] Authenticating user token {{(pid=80145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 06 13:20:11.020352 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-3b08c664-c92e-468d-ae79-942138e72ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:20:11.139489 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08c664-c92e-468d-ae79-942138e72ecd admin admin] Validating token access rules against request {{(pid=80145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 06 13:20:11.197040 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f801be-0e25-4387-b833-051558e887bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2244655ba304b9c980b5e68772eefc0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13fba33c46cf40bbb15a08c690b4c98c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 06 13:20:11.197653 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f801be-0e25-4387-b833-051558e887bc service placement] REQUEST_METHOD: `GET` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 06 13:20:11.197934 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f801be-0e25-4387-b833-051558e887bc service placement] SCRIPT_NAME: `/identity` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 06 13:20:11.198209 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f801be-0e25-4387-b833-051558e887bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 06 13:20:11.199964 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.fernet_utils [None req-d9f801be-0e25-4387-b833-051558e887bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 06 13:20:11.210990 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f801be-0e25-4387-b833-051558e887bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 06 13:20:11.212598 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f801be-0e25-4387-b833-051558e887bc service placement] RBAC: Authorization granted {{(pid=80145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 06 13:20:11.222419 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: [pid: 80145|app: 0|req: 1466/2968] 10.222.0.20 () {60 vars in 1365 bytes} [Sat Aug 6 13:20:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 06 13:20:21.218194 dv-cinder-851996-3-iscsi devstack@keystone.service[80145]: DEBUG dbcounter [-] [80145] Writing DB stats keystone:SELECT=80 {{(pid=80145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}